15/01/29 11:47:54 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces 15/01/29 11:47:54 INFO Configuration.deprecation: mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize 15/01/29 11:47:54 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative 15/01/29 11:47:54 INFO Configuration.deprecation: mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node 15/01/29 11:47:54 INFO Configuration.deprecation: mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive 15/01/29 11:47:54 INFO Configuration.deprecation: mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack 15/01/29 11:47:54 INFO Configuration.deprecation: mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize 15/01/29 11:47:54 INFO Configuration.deprecation: mapred.committer.job.setup.cleanup.needed is deprecated. Instead, use mapreduce.job.committer.setup.cleanup.needed 15/01/29 11:47:55 WARN conf.HiveConf: HiveConf of name hive.mapjoin.optimized.keys does not exist 15/01/29 11:47:55 WARN conf.HiveConf: HiveConf of name hive.mapjoin.lazy.hashtable does not exist 15/01/29 11:47:55 [main]: DEBUG common.LogUtils: Using hive-site.xml found on CLASSPATH at /home/zhangyun/hive-site.xml 15/01/29 11:47:55 [main]: WARN conf.HiveConf: HiveConf of name hive.mapjoin.optimized.keys does not exist 15/01/29 11:47:55 [main]: WARN conf.HiveConf: HiveConf of name hive.mapjoin.lazy.hashtable does not exist Logging initialized using configuration in jar:file:/home/zhangyun/tools/hive/packaging/target/apache-hive-0.15.0-SNAPSHOT-bin/apache-hive-0.15.0-SNAPSHOT-bin/lib/hive-common-0.15.0-SNAPSHOT.jar!/hive-log4j.properties 15/01/29 11:47:55 [main]: INFO SessionState: Logging initialized using configuration in jar:file:/home/zhangyun/tools/hive/packaging/target/apache-hive-0.15.0-SNAPSHOT-bin/apache-hive-0.15.0-SNAPSHOT-bin/lib/hive-common-0.15.0-SNAPSHOT.jar!/hive-log4j.properties 15/01/29 11:47:55 [main]: DEBUG parse.VariableSubstitution: Substitution is on: hive 15/01/29 11:47:55 [main]: DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate of successful kerberos logins and latency (milliseconds)], about=, type=DEFAULT, always=false, sampleName=Ops) 15/01/29 11:47:55 [main]: DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate of failed kerberos logins and latency (milliseconds)], about=, type=DEFAULT, always=false, sampleName=Ops) 15/01/29 11:47:55 [main]: DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/zhangyun/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/zhangyun/tools/hive/packaging/target/apache-hive-0.15.0-SNAPSHOT-bin/apache-hive-0.15.0-SNAPSHOT-bin/lib/hive-jdbc-0.15.0-SNAPSHOT-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 15/01/29 11:47:55 [main]: DEBUG security.Groups: Creating new Groups object 15/01/29 11:47:55 [main]: DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library... Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /home/zhangyun/hadoop/lib/native/libhadoop.so which might have disabled stack guard. The VM will try to fix the stack guard now. It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'. 15/01/29 11:47:55 [main]: DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: /home/zhangyun/hadoop/lib/native/libhadoop.so: /home/zhangyun/hadoop/lib/native/libhadoop.so: wrong ELF class: ELFCLASS32 (Possible cause: architecture word width mismatch) 15/01/29 11:47:55 [main]: DEBUG util.NativeCodeLoader: java.library.path=/home/zhangyun/hadoop/lib/native 15/01/29 11:47:55 [main]: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 15/01/29 11:47:55 [main]: DEBUG security.JniBasedUnixGroupsMappingWithFallback: Falling back to shell based 15/01/29 11:47:55 [main]: DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping 15/01/29 11:47:55 [main]: DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000 15/01/29 11:47:55 [main]: DEBUG security.UserGroupInformation: hadoop login 15/01/29 11:47:55 [main]: DEBUG security.UserGroupInformation: hadoop login commit 15/01/29 11:47:55 [main]: DEBUG security.UserGroupInformation: using local user:UnixPrincipal: zhangyun 15/01/29 11:47:55 [main]: DEBUG security.UserGroupInformation: UGI loginUser:zhangyun (auth:SIMPLE) 15/01/29 11:47:55 [main]: INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.validateConstraints value null from jpox.properties with false 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.cache.level2 value null from jpox.properties with false 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding javax.jdo.option.ConnectionUserName value null from jpox.properties with APP 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.connectionPoolingType value null from jpox.properties with BONECP 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.fixedDatastore value null from jpox.properties with false 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding javax.jdo.option.ConnectionDriverName value null from jpox.properties with org.apache.derby.jdbc.EmbeddedDriver 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.validateColumns value null from jpox.properties with false 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.validateTables value null from jpox.properties with false 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.cache.level2.type value null from jpox.properties with none 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.plugin.pluginRegistryBundleCheck value null from jpox.properties with LOG 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.autoStartMechanismMode value null from jpox.properties with checked 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.autoCreateSchema value null from jpox.properties with true 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding javax.jdo.option.NonTransactionalRead value null from jpox.properties with true 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.rdbms.useLegacyNativeValueStrategy value null from jpox.properties with true 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.identifierFactory value null from jpox.properties with datanucleus1 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding javax.jdo.option.ConnectionURL value null from jpox.properties with jdbc:derby:;databaseName=metastore_db;create=true 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding javax.jdo.option.DetachAllOnCommit value null from jpox.properties with true 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding hive.metastore.integral.jdo.pushdown value null from jpox.properties with false 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.storeManagerType value null from jpox.properties with rdbms 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.transactionIsolation value null from jpox.properties with read-committed 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding javax.jdo.PersistenceManagerFactoryClass value null from jpox.properties with org.datanucleus.api.jdo.JDOPersistenceManagerFactory 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: Overriding javax.jdo.option.Multithreaded value null from jpox.properties with true 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: datanucleus.rdbms.useLegacyNativeValueStrategy = true 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: hive.metastore.integral.jdo.pushdown = false 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: datanucleus.autoStartMechanismMode = checked 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: javax.jdo.option.Multithreaded = true 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: datanucleus.identifierFactory = datanucleus1 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: datanucleus.transactionIsolation = read-committed 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: datanucleus.validateTables = false 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: javax.jdo.option.ConnectionURL = jdbc:derby:;databaseName=metastore_db;create=true 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: javax.jdo.option.DetachAllOnCommit = true 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: javax.jdo.option.NonTransactionalRead = true 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: datanucleus.fixedDatastore = false 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: datanucleus.validateConstraints = false 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: javax.jdo.option.ConnectionDriverName = org.apache.derby.jdbc.EmbeddedDriver 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: javax.jdo.option.ConnectionUserName = APP 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: datanucleus.validateColumns = false 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: datanucleus.cache.level2 = false 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: datanucleus.plugin.pluginRegistryBundleCheck = LOG 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: datanucleus.cache.level2.type = none 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: javax.jdo.PersistenceManagerFactoryClass = org.datanucleus.api.jdo.JDOPersistenceManagerFactory 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: datanucleus.autoCreateSchema = true 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: datanucleus.storeManagerType = rdbms 15/01/29 11:47:55 [main]: DEBUG metastore.ObjectStore: datanucleus.connectionPoolingType = BONECP 15/01/29 11:47:55 [main]: INFO metastore.ObjectStore: ObjectStore, initialize called 15/01/29 11:47:56 [main]: DEBUG bonecp.BoneCPDataSource: JDBC URL = jdbc:derby:;databaseName=metastore_db;create=true, Username = APP, partitions = 1, max (per partition) = 10, min (per partition) = 0, idle max age = 60 min, idle test period = 240 min, strategy = DEFAULT 15/01/29 11:47:58 [main]: WARN conf.HiveConf: HiveConf of name hive.mapjoin.optimized.keys does not exist 15/01/29 11:47:58 [main]: WARN conf.HiveConf: HiveConf of name hive.mapjoin.lazy.hashtable does not exist 15/01/29 11:47:58 [main]: INFO metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" 15/01/29 11:47:58 [main]: DEBUG bonecp.BoneCPDataSource: JDBC URL = jdbc:derby:;databaseName=metastore_db;create=true, Username = APP, partitions = 1, max (per partition) = 10, min (per partition) = 0, idle max age = 60 min, idle test period = 240 min, strategy = DEFAULT 15/01/29 11:47:58 [main]: DEBUG metastore.MetaStoreDirectSql: MySql check failed, assuming we are not on MySql: Lexical error at line 1, column 5. Encountered: "@" (64), after : "". 15/01/29 11:47:58 [main]: DEBUG metastore.MetaStoreDirectSql: Oracle check failed, assuming we are not on Oracle: Lexical error at line 1, column 22. Encountered: "$" (36), after : "". 15/01/29 11:47:58 [main]: DEBUG metastore.MetaStoreDirectSql: MSSQL check failed, assuming we are not on MSSQL: Lexical error at line 1, column 8. Encountered: "@" (64), after : "". 15/01/29 11:48:00 [main]: INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: RawStore: org.apache.hadoop.hive.metastore.ObjectStore@497c12f2, with PersistenceManager: org.datanucleus.api.jdo.JDOPersistenceManager@2185cbf created in the thread with id: 1 15/01/29 11:48:00 [main]: INFO metastore.ObjectStore: Initialized ObjectStore 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getMSchemaVersion(ObjectStore.java:6650) 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 0, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.getMSchemaVersion(ObjectStore.java:6664) 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.start(ObjectStore.java:2388) 15/01/29 11:48:00 [main]: DEBUG metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[hdfs://10.3.1.76:29000/home/zhangyun/runtime/hive/warehouse] desc [Default Hive database] owner [null] ownertype [null] 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 0, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.commit(ObjectStore.java:2425) 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: db details for db default retrieved using SQL in 16.51472ms 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.addRole(ObjectStore.java:3204) 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 2, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getMRole(ObjectStore.java:3511) 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 1, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.getMRole(ObjectStore.java:3517) 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Rollback transaction, isActive: true at: org.apache.hadoop.hive.metastore.ObjectStore.addRole(ObjectStore.java:3217) 15/01/29 11:48:00 [main]: DEBUG metastore.HiveMetaStore: admin role already exists InvalidObjectException(message:Role admin already exists.) at org.apache.hadoop.hive.metastore.ObjectStore.addRole(ObjectStore.java:3207) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:98) at com.sun.proxy.$Proxy7.addRole(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultRoles_core(HiveMetaStore.java:640) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultRoles(HiveMetaStore.java:629) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:447) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:66) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72) at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5574) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:188) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:73) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1479) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:64) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:74) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2707) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2726) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:431) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) 15/01/29 11:48:00 [main]: INFO metastore.HiveMetaStore: Added admin role in metastore 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.addRole(ObjectStore.java:3204) 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 2, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getMRole(ObjectStore.java:3511) 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 1, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.getMRole(ObjectStore.java:3517) 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Rollback transaction, isActive: true at: org.apache.hadoop.hive.metastore.ObjectStore.addRole(ObjectStore.java:3217) 15/01/29 11:48:00 [main]: DEBUG metastore.HiveMetaStore: public role already exists InvalidObjectException(message:Role public already exists.) at org.apache.hadoop.hive.metastore.ObjectStore.addRole(ObjectStore.java:3207) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:98) at com.sun.proxy.$Proxy7.addRole(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultRoles_core(HiveMetaStore.java:649) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultRoles(HiveMetaStore.java:629) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:447) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:66) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72) at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5574) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:188) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:73) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1479) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:64) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:74) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2707) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2726) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:431) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) 15/01/29 11:48:00 [main]: INFO metastore.HiveMetaStore: Added public role in metastore 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.grantPrivileges(ObjectStore.java:3896) 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 2, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getMRole(ObjectStore.java:3511) 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 1, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.getMRole(ObjectStore.java:3517) 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 2, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.listPrincipalGlobalGrants(ObjectStore.java:4381) 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 1, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.listPrincipalGlobalGrants(ObjectStore.java:4391) 15/01/29 11:48:00 [main]: DEBUG metastore.ObjectStore: Rollback transaction, isActive: true at: org.apache.hadoop.hive.metastore.ObjectStore.grantPrivileges(ObjectStore.java:4099) 15/01/29 11:48:00 [main]: DEBUG metastore.HiveMetaStore: Failed while granting global privs to admin InvalidObjectException(message:All is already granted by admin) at org.apache.hadoop.hive.metastore.ObjectStore.grantPrivileges(ObjectStore.java:3932) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:98) at com.sun.proxy.$Proxy7.grantPrivileges(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultRoles_core(HiveMetaStore.java:663) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultRoles(HiveMetaStore.java:629) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:447) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:66) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72) at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5574) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:188) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:73) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1479) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:64) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:74) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2707) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2726) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:431) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) 15/01/29 11:48:00 [main]: INFO metastore.HiveMetaStore: No user is added in admin role, since config is empty 15/01/29 11:48:00 [main]: DEBUG hdfs.BlockReaderLocal: dfs.client.use.legacy.blockreader.local = false 15/01/29 11:48:00 [main]: DEBUG hdfs.BlockReaderLocal: dfs.client.read.shortcircuit = false 15/01/29 11:48:00 [main]: DEBUG hdfs.BlockReaderLocal: dfs.client.domain.socket.data.traffic = false 15/01/29 11:48:00 [main]: DEBUG hdfs.BlockReaderLocal: dfs.domain.socket.path = 15/01/29 11:48:00 [main]: DEBUG impl.MetricsSystemImpl: StartupProgress, NameNode startup progress 15/01/29 11:48:00 [main]: DEBUG retry.RetryUtils: multipleLinearRandomRetry = null 15/01/29 11:48:00 [main]: DEBUG ipc.Server: rpcKind=RPC_PROTOCOL_BUFFER, rpcRequestWrapperClass=class org.apache.hadoop.ipc.ProtobufRpcEngine$RpcRequestWrapper, rpcInvoker=org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker@62b3a369 15/01/29 11:48:01 [main]: DEBUG hdfs.BlockReaderLocal: Both short-circuit local reads and UNIX domain socket are disabled. 15/01/29 11:48:01 [main]: DEBUG ipc.Client: The ping interval is 60000 ms. 15/01/29 11:48:01 [main]: DEBUG ipc.Client: Connecting to /10.3.1.76:29000 15/01/29 11:48:01 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun: starting, having connections 1 15/01/29 11:48:01 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun sending #0 15/01/29 11:48:01 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun got value #0 15/01/29 11:48:01 [main]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 40ms 15/01/29 11:48:01 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun sending #1 15/01/29 11:48:01 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun got value #1 15/01/29 11:48:01 [main]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 2ms 15/01/29 11:48:01 [main]: DEBUG session.SessionState: HDFS root scratch dir: /home/zhangyun/runtime/tmp/hive-0.15, permission: rwx-wx-wx 15/01/29 11:48:01 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun sending #2 15/01/29 11:48:01 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun got value #2 15/01/29 11:48:01 [main]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 1ms 15/01/29 11:48:01 [main]: INFO session.SessionState: Created local directory: /home/zhangyun/runtime/tmp/hive-0.15 15/01/29 11:48:01 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun sending #3 15/01/29 11:48:01 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun got value #3 15/01/29 11:48:01 [main]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 1ms 15/01/29 11:48:01 [main]: DEBUG hdfs.DFSClient: /home/zhangyun/runtime/tmp/hive-0.15/zhangyun/537b16f4-4652-403b-9710-8bced8ba3c42: masked=rwx------ 15/01/29 11:48:01 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun sending #4 15/01/29 11:48:01 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun got value #4 15/01/29 11:48:01 [main]: DEBUG ipc.ProtobufRpcEngine: Call: mkdirs took 10ms 15/01/29 11:48:01 [main]: INFO session.SessionState: Created HDFS directory: /home/zhangyun/runtime/tmp/hive-0.15/zhangyun/537b16f4-4652-403b-9710-8bced8ba3c42 15/01/29 11:48:01 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun sending #5 15/01/29 11:48:01 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun got value #5 15/01/29 11:48:01 [main]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 1ms 15/01/29 11:48:01 [main]: INFO session.SessionState: Created local directory: /home/zhangyun/runtime/tmp/hive-0.15/537b16f4-4652-403b-9710-8bced8ba3c42 15/01/29 11:48:01 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun sending #6 15/01/29 11:48:01 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun got value #6 15/01/29 11:48:01 [main]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 1ms 15/01/29 11:48:01 [main]: DEBUG hdfs.DFSClient: /home/zhangyun/runtime/tmp/hive-0.15/zhangyun/537b16f4-4652-403b-9710-8bced8ba3c42/_tmp_space.db: masked=rwx------ 15/01/29 11:48:01 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun sending #7 15/01/29 11:48:01 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun got value #7 15/01/29 11:48:01 [main]: DEBUG ipc.ProtobufRpcEngine: Call: mkdirs took 7ms 15/01/29 11:48:01 [main]: INFO session.SessionState: Created HDFS directory: /home/zhangyun/runtime/tmp/hive-0.15/zhangyun/537b16f4-4652-403b-9710-8bced8ba3c42/_tmp_space.db 15/01/29 11:48:01 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun sending #8 15/01/29 11:48:01 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun got value #8 15/01/29 11:48:01 [main]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 1ms 15/01/29 11:48:01 [main]: INFO session.SessionState: No Tez session required at this point. hive.execution.engine=mr. 15/01/29 11:48:01 [main]: DEBUG parse.VariableSubstitution: Substitution is on: true 15/01/29 11:48:01 [main]: INFO log.PerfLogger: 15/01/29 11:48:01 [main]: INFO log.PerfLogger: 15/01/29 11:48:01 [main]: INFO log.PerfLogger: 15/01/29 11:48:01 [main]: DEBUG parse.VariableSubstitution: Substitution is on: select l_orderkey, sum(l_extendedprice*(1-l_discount)) as revenue, o_orderdate, o_shippriority from lineitem l join orders o on l.l_orderkey = o.o_orderkey join customer c on c.c_mktsegment = 'BUILDING' and c.c_custkey = o.o_custkey where o_orderdate < '1995-03-15' and l_shipdate > '1995-03-15' group by l_orderkey, o_orderdate, o_shippriority order by revenue desc, o_orderdate limit 10 15/01/29 11:48:01 [main]: INFO log.PerfLogger: 15/01/29 11:48:01 [main]: INFO parse.ParseDriver: Parsing command: select l_orderkey, sum(l_extendedprice*(1-l_discount)) as revenue, o_orderdate, o_shippriority from lineitem l join orders o on l.l_orderkey = o.o_orderkey join customer c on c.c_mktsegment = 'BUILDING' and c.c_custkey = o.o_custkey where o_orderdate < '1995-03-15' and l_shipdate > '1995-03-15' group by l_orderkey, o_orderdate, o_shippriority order by revenue desc, o_orderdate limit 10 15/01/29 11:48:01 [main]: INFO parse.ParseDriver: Parse Completed 15/01/29 11:48:01 [main]: INFO log.PerfLogger: 15/01/29 11:48:01 [main]: DEBUG ql.Driver: Encoding valid txns info 9223372036854775807: 15/01/29 11:48:01 [main]: INFO log.PerfLogger: 15/01/29 11:48:01 [main]: INFO parse.CalcitePlanner: Starting Semantic Analysis 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:01 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: DEBUG exec.FunctionRegistry: Looking up GenericUDAF: sum 15/01/29 11:48:02 [main]: INFO parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 15/01/29 11:48:02 [main]: INFO parse.CalcitePlanner: Get metadata for source tables 15/01/29 11:48:02 [main]: INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 15/01/29 11:48:02 [main]: INFO HiveMetaStore.audit: ugi=zhangyun ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 15/01/29 11:48:02 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:909) 15/01/29 11:48:02 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 2, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:972) 15/01/29 11:48:02 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 1, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:980) 15/01/29 11:48:02 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 0, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:911) 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=orders 15/01/29 11:48:02 [main]: INFO HiveMetaStore.audit: ugi=zhangyun ip=unknown-ip-addr cmd=get_table : db=default tbl=orders 15/01/29 11:48:02 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:909) 15/01/29 11:48:02 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 2, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:972) 15/01/29 11:48:02 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 1, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:980) 15/01/29 11:48:02 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 0, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:911) 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=customer 15/01/29 11:48:02 [main]: INFO HiveMetaStore.audit: ugi=zhangyun ip=unknown-ip-addr cmd=get_table : db=default tbl=customer 15/01/29 11:48:02 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:909) 15/01/29 11:48:02 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 2, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:972) 15/01/29 11:48:02 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 1, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:980) 15/01/29 11:48:02 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 0, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:911) 15/01/29 11:48:02 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:02 [main]: INFO parse.CalcitePlanner: Get metadata for subqueries 15/01/29 11:48:02 [main]: INFO parse.CalcitePlanner: Get metadata for destination tables 15/01/29 11:48:02 [main]: DEBUG hdfs.DFSClient: /home/zhangyun/runtime/tmp/hive-0.15/zhangyun/537b16f4-4652-403b-9710-8bced8ba3c42/hive_2015-01-29_11-48-01_394_4926730040135186240-1: masked=rwx------ 15/01/29 11:48:02 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun sending #9 15/01/29 11:48:02 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun got value #9 15/01/29 11:48:02 [main]: DEBUG ipc.ProtobufRpcEngine: Call: mkdirs took 18ms 15/01/29 11:48:02 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun sending #10 15/01/29 11:48:02 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun got value #10 15/01/29 11:48:02 [main]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 2ms 15/01/29 11:48:02 [main]: INFO ql.Context: New scratch dir is hdfs://10.3.1.76:29000/home/zhangyun/runtime/tmp/hive-0.15/zhangyun/537b16f4-4652-403b-9710-8bced8ba3c42/hive_2015-01-29_11-48-01_394_4926730040135186240-1 15/01/29 11:48:02 [main]: INFO parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 15/01/29 11:48:03 [main]: DEBUG hive.log: DDL: struct lineitem { i32 l_orderkey, i32 l_partkey, i32 l_suppkey, i32 l_linenumber, double l_quantity, double l_extendedprice, double l_discount, double l_tax, string l_returnflag, string l_linestatus, string l_shipdate, string l_commitdate, string l_receiptdate, string l_shipinstruct, string l_shipmode, string l_comment} 15/01/29 11:48:03 [main]: DEBUG lazy.LazySimpleSerDe: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe initialized with: columnNames=[l_orderkey, l_partkey, l_suppkey, l_linenumber, l_quantity, l_extendedprice, l_discount, l_tax, l_returnflag, l_linestatus, l_shipdate, l_commitdate, l_receiptdate, l_shipinstruct, l_shipmode, l_comment] columnTypes=[int, int, int, int, double, double, double, double, string, string, string, string, string, string, string, string] separator=[[B@171ef241] nullstring=\N lastColumnTakesRest=false Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveTableScan#0 15/01/29 11:48:03 [main]: DEBUG hive.log: DDL: struct orders { i32 o_orderkey, i32 o_custkey, string o_orderstatus, double o_totalprice, string o_orderdate, string o_orderpriority, string o_clerk, i32 o_shippriority, string o_comment} 15/01/29 11:48:03 [main]: DEBUG lazy.LazySimpleSerDe: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe initialized with: columnNames=[o_orderkey, o_custkey, o_orderstatus, o_totalprice, o_orderdate, o_orderpriority, o_clerk, o_shippriority, o_comment] columnTypes=[int, int, string, double, string, string, string, int, string] separator=[[B@7d0afdc2] nullstring=\N lastColumnTakesRest=false Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveTableScan#1 15/01/29 11:48:03 [main]: DEBUG hive.log: DDL: struct customer { i32 c_custkey, string c_name, string c_address, i32 c_nationkey, string c_phone, double c_acctbal, string c_mktsegment, string c_comment} 15/01/29 11:48:03 [main]: DEBUG lazy.LazySimpleSerDe: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe initialized with: columnNames=[c_custkey, c_name, c_address, c_nationkey, c_phone, c_acctbal, c_mktsegment, c_comment] columnTypes=[int, string, string, int, string, double, string, string] separator=[[B@5757f4f] nullstring=\N lastColumnTakesRest=false Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveTableScan#2 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#3 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#4 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#5 15/01/29 11:48:03 [main]: DEBUG exec.FunctionRegistry: Looking up GenericUDAF: sum 15/01/29 11:48:03 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:03 [main]: DEBUG exec.FunctionRegistry: Method did match: passed = [int] accepted = [decimal(38,18)] method = public org.apache.hadoop.hive.serde2.io.DoubleWritable org.apache.hadoop.hive.ql.udf.UDFToDouble.evaluate(org.apache.hadoop.hive.serde2.io.HiveDecimalWritable) 15/01/29 11:48:03 [main]: DEBUG exec.FunctionRegistry: Method did match: passed = [int] accepted = [bigint] method = public org.apache.hadoop.hive.serde2.io.DoubleWritable org.apache.hadoop.hive.ql.udf.UDFToDouble.evaluate(org.apache.hadoop.io.LongWritable) 15/01/29 11:48:03 [main]: DEBUG exec.FunctionRegistry: Method did match: passed = [int] accepted = [float] method = public org.apache.hadoop.hive.serde2.io.DoubleWritable org.apache.hadoop.hive.ql.udf.UDFToDouble.evaluate(org.apache.hadoop.io.FloatWritable) 15/01/29 11:48:03 [main]: DEBUG exec.FunctionRegistry: Method did match: passed = [int] accepted = [string] method = public org.apache.hadoop.hive.serde2.io.DoubleWritable org.apache.hadoop.hive.ql.udf.UDFToDouble.evaluate(org.apache.hadoop.io.Text) 15/01/29 11:48:03 [main]: DEBUG exec.FunctionRegistry: Method didn't match: passed = [int] accepted = [timestamp] method = public org.apache.hadoop.hive.serde2.io.DoubleWritable org.apache.hadoop.hive.ql.udf.UDFToDouble.evaluate(org.apache.hadoop.hive.serde2.io.TimestampWritable) 15/01/29 11:48:03 [main]: DEBUG exec.FunctionRegistry: Method did match: passed = [int] accepted = [int] method = public org.apache.hadoop.hive.serde2.io.DoubleWritable org.apache.hadoop.hive.ql.udf.UDFToDouble.evaluate(org.apache.hadoop.io.IntWritable) 15/01/29 11:48:03 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:03 [main]: DEBUG exec.Utilities: Use session specified class loader 15/01/29 11:48:03 [main]: DEBUG exec.Utilities: Use session specified class loader Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#6 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveAggregate#7 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#8 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveSort#9 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveSort#10 15/01/29 11:48:03 [main]: DEBUG parse.CalcitePlanner: Created Plan for Query Block null Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#11 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#12 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#13 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#14 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#15 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#16 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#17 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#18 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#19 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#20 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#21 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveAggregate#22 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#23 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#24 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#25 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveSort#26 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#27 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveSort#28 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#29 15/01/29 11:48:03 [main]: INFO log.PerfLogger: 15/01/29 11:48:03 [main]: INFO log.PerfLogger: 15/01/29 11:48:03 [main]: INFO metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=lineitem 15/01/29 11:48:03 [main]: INFO HiveMetaStore.audit: ugi=zhangyun ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=lineitem 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.start(ObjectStore.java:2388) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 2, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:972) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 1, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:980) 15/01/29 11:48:03 [main]: DEBUG metastore.MetaStoreDirectSql: Direct SQL query in 8.171591ms + 16.656721ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (?)] 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 0, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.commit(ObjectStore.java:2425) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: statistics for 1 columns retrieved using SQL in 47.470434ms 15/01/29 11:48:03 [main]: INFO log.PerfLogger: 15/01/29 11:48:03 [main]: INFO log.PerfLogger: 15/01/29 11:48:03 [main]: INFO metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=orders 15/01/29 11:48:03 [main]: INFO HiveMetaStore.audit: ugi=zhangyun ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=orders 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.start(ObjectStore.java:2388) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 2, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:972) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 1, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:980) 15/01/29 11:48:03 [main]: DEBUG metastore.MetaStoreDirectSql: Direct SQL query in 9.596403ms + 2.659849ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (?,?)] 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 0, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.commit(ObjectStore.java:2425) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: statistics for 2 columns retrieved using SQL in 32.890445ms 15/01/29 11:48:03 [main]: INFO metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=orders 15/01/29 11:48:03 [main]: INFO HiveMetaStore.audit: ugi=zhangyun ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=orders 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.start(ObjectStore.java:2388) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 2, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:972) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 1, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:980) 15/01/29 11:48:03 [main]: DEBUG metastore.MetaStoreDirectSql: Direct SQL query in 0.980388ms + 0.110888ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (?)] 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 0, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.commit(ObjectStore.java:2425) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: statistics for 1 columns retrieved using SQL in 20.839246ms 15/01/29 11:48:03 [main]: INFO log.PerfLogger: 15/01/29 11:48:03 [main]: INFO log.PerfLogger: 15/01/29 11:48:03 [main]: INFO metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=orders 15/01/29 11:48:03 [main]: INFO HiveMetaStore.audit: ugi=zhangyun ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=orders 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.start(ObjectStore.java:2388) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 2, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:972) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 1, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:980) 15/01/29 11:48:03 [main]: DEBUG metastore.MetaStoreDirectSql: Direct SQL query in 0.99888ms + 0.124402ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (?)] 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 0, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.commit(ObjectStore.java:2425) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: statistics for 1 columns retrieved using SQL in 20.788515ms 15/01/29 11:48:03 [main]: INFO metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=customer 15/01/29 11:48:03 [main]: INFO HiveMetaStore.audit: ugi=zhangyun ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=customer 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.start(ObjectStore.java:2388) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 2, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:972) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 1, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:980) 15/01/29 11:48:03 [main]: DEBUG metastore.MetaStoreDirectSql: Direct SQL query in 0.920487ms + 0.106664ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (?)] 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 0, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.commit(ObjectStore.java:2425) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: statistics for 1 columns retrieved using SQL in 20.138279ms 15/01/29 11:48:03 [main]: INFO metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=customer 15/01/29 11:48:03 [main]: INFO HiveMetaStore.audit: ugi=zhangyun ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=customer 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.start(ObjectStore.java:2388) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 2, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:972) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 1, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:980) 15/01/29 11:48:03 [main]: DEBUG metastore.MetaStoreDirectSql: Direct SQL query in 0.945898ms + 0.109102ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (?)] 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 0, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.commit(ObjectStore.java:2425) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: statistics for 1 columns retrieved using SQL in 19.457526ms Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.hep.HepPlanner dumpGraph FINER: Breadth-first from root: { HepRelVertex#29 = rel#28:HiveSort.HIVE.[](input=HepRelVertex#27,fetch=10), rowcount=1.0, cumulative cost={36.0 rows, 2.0 cpu, 0.0 io} HepRelVertex#27 = rel#26:HiveSort.HIVE.[1 DESC, 2](input=HepRelVertex#25,sort0=$1,sort1=$2,dir0=DESC,dir1=ASC), rowcount=1.0, cumulative cost={20.0 rows, 1.0 cpu, 0.0 io} HepRelVertex#25 = rel#24:HiveProject.HIVE.[](input=HepRelVertex#23,l_orderkey=$0,revenue=$3,o_orderdate=$1,o_shippriority=$2), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#23 = rel#22:HiveAggregate.HIVE.[](input=HepRelVertex#21,group={0, 1, 2},agg#0=sum($3)), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#21 = rel#20:HiveProject.HIVE.[](input=HepRelVertex#19,$f0=$0,$f1=$23,$f2=$26,$f3=*($5, -(CAST(1):DOUBLE, $6))), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#19 = rel#18:HiveFilter.HIVE.[](input=HepRelVertex#17,condition=AND(<($23, '1995-03-15'), >($10, '1995-03-15'))), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#17 = rel#16:HiveJoin.HIVE.[](left=HepRelVertex#14,right=HepRelVertex#15,condition=AND(=($37, 'BUILDING'), =($31, $20)),joinType=inner), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#14 = rel#13:HiveJoin.HIVE.[](left=HepRelVertex#11,right=HepRelVertex#12,condition==($0, $19),joinType=inner), rowcount=1.0, cumulative cost={2.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#15 = rel#2:HiveTableScan.HIVE.[](table=[default.customer]), rowcount=1.0, cumulative cost={0} HepRelVertex#11 = rel#0:HiveTableScan.HIVE.[](table=[default.lineitem]), rowcount=1.0, cumulative cost={0} HepRelVertex#12 = rel#1:HiveTableScan.HIVE.[](table=[default.orders]), rowcount=1.0, cumulative cost={0} } Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.hep.HepPlanner executeInstruction FINEST: Setting match limit to TOP_DOWN Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.hep.HepPlanner applyRules FINEST: Applying rule set [SemiJoinJoinTransposeRule, SemiJoinFilterTransposeRule, SemiJoinProjectTransposeRule] Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#28:HiveSort.HIVE.[](input=HepRelVertex#27,fetch=10) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#26:HiveSort.HIVE.[1 DESC, 2](input=HepRelVertex#25,sort0=$1,sort1=$2,dir0=DESC,dir1=ASC) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#24:HiveProject.HIVE.[](input=HepRelVertex#23,l_orderkey=$0,revenue=$3,o_orderdate=$1,o_shippriority=$2) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#22:HiveAggregate.HIVE.[](input=HepRelVertex#21,group={0, 1, 2},agg#0=sum($3)) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#20:HiveProject.HIVE.[](input=HepRelVertex#19,$f0=$0,$f1=$23,$f2=$26,$f3=*($5, -(CAST(1):DOUBLE, $6))) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#18:HiveFilter.HIVE.[](input=HepRelVertex#17,condition=AND(<($23, '1995-03-15'), >($10, '1995-03-15'))) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#16:HiveJoin.HIVE.[](left=HepRelVertex#14,right=HepRelVertex#15,condition=AND(=($37, 'BUILDING'), =($31, $20)),joinType=inner) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#13:HiveJoin.HIVE.[](left=HepRelVertex#11,right=HepRelVertex#12,condition==($0, $19),joinType=inner) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#0:HiveTableScan.HIVE.[](table=[default.lineitem]) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#1:HiveTableScan.HIVE.[](table=[default.orders]) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#2:HiveTableScan.HIVE.[](table=[default.customer]) Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#30 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#31 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#32 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#33 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#34 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#35 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#36 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#37 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#38 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#39 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#40 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveAggregate#41 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#42 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#43 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#44 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveSort#45 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#46 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveSort#47 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#48 Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.hep.HepPlanner dumpGraph FINER: Breadth-first from root: { HepRelVertex#48 = rel#47:HiveSort.HIVE.[](input=HepRelVertex#46,fetch=10), rowcount=1.0, cumulative cost={36.0 rows, 2.0 cpu, 0.0 io} HepRelVertex#46 = rel#45:HiveSort.HIVE.[1 DESC, 2](input=HepRelVertex#44,sort0=$1,sort1=$2,dir0=DESC,dir1=ASC), rowcount=1.0, cumulative cost={20.0 rows, 1.0 cpu, 0.0 io} HepRelVertex#44 = rel#43:HiveProject.HIVE.[](input=HepRelVertex#42,l_orderkey=$0,revenue=$3,o_orderdate=$1,o_shippriority=$2), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#42 = rel#41:HiveAggregate.HIVE.[](input=HepRelVertex#40,group={0, 1, 2},agg#0=sum($3)), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#40 = rel#39:HiveProject.HIVE.[](input=HepRelVertex#38,$f0=$0,$f1=$23,$f2=$26,$f3=*($5, -(CAST(1):DOUBLE, $6))), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#38 = rel#37:HiveFilter.HIVE.[](input=HepRelVertex#36,condition=AND(<($23, '1995-03-15'), >($10, '1995-03-15'))), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#36 = rel#35:HiveJoin.HIVE.[](left=HepRelVertex#33,right=HepRelVertex#34,condition=AND(=($37, 'BUILDING'), =($31, $20)),joinType=inner), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#33 = rel#32:HiveJoin.HIVE.[](left=HepRelVertex#30,right=HepRelVertex#31,condition==($0, $19),joinType=inner), rowcount=1.0, cumulative cost={2.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#34 = rel#2:HiveTableScan.HIVE.[](table=[default.customer]), rowcount=1.0, cumulative cost={0} HepRelVertex#30 = rel#0:HiveTableScan.HIVE.[](table=[default.lineitem]), rowcount=1.0, cumulative cost={0} HepRelVertex#31 = rel#1:HiveTableScan.HIVE.[](table=[default.orders]), rowcount=1.0, cumulative cost={0} } Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.hep.HepPlanner executeInstruction FINEST: Setting match limit to TOP_DOWN Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.hep.HepPlanner applyRules FINEST: Applying rule set [ReduceExpressionsRule(Project), ReduceExpressionsRule(Filter), ReduceExpressionsRule(Join), HiveFilterProjectTransposeRule, HiveFilterSetOpTransposeRule, FilterMergeRule, PushFilterRule:HiveFilterJoinRule:no-filter, PushFilterRule:HiveFilterJoinRule:filter, FilterAggregateTransposeRule] Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#0: Apply rule [PushFilterRule:HiveFilterJoinRule:filter] to [rel#37:HiveFilter.HIVE.[](input=HepRelVertex#36,condition=AND(<($23, '1995-03-15'), >($10, '1995-03-15'))), rel#35:HiveJoin.HIVE.[](left=HepRelVertex#33,right=HepRelVertex#34,condition=AND(=($37, 'BUILDING'), =($31, $20)),joinType=inner)] Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#49 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#50 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#51 Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyTransformation FINE: call#0: Rule PushFilterRule:HiveFilterJoinRule:filter arguments [rel#37:HiveFilter.HIVE.[](input=HepRelVertex#36,condition=AND(<($23, '1995-03-15'), >($10, '1995-03-15'))), rel#35:HiveJoin.HIVE.[](left=HepRelVertex#33,right=HepRelVertex#34,condition=AND(=($37, 'BUILDING'), =($31, $20)),joinType=inner)] produced HiveJoin#51 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#52 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#53 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#54 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#55 Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.hep.HepPlanner dumpGraph FINER: Breadth-first from root: { HepRelVertex#48 = rel#47:HiveSort.HIVE.[](input=HepRelVertex#46,fetch=10), rowcount=1.0, cumulative cost={36.0 rows, 2.0 cpu, 0.0 io} HepRelVertex#46 = rel#45:HiveSort.HIVE.[1 DESC, 2](input=HepRelVertex#44,sort0=$1,sort1=$2,dir0=DESC,dir1=ASC), rowcount=1.0, cumulative cost={20.0 rows, 1.0 cpu, 0.0 io} HepRelVertex#44 = rel#43:HiveProject.HIVE.[](input=HepRelVertex#42,l_orderkey=$0,revenue=$3,o_orderdate=$1,o_shippriority=$2), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#42 = rel#41:HiveAggregate.HIVE.[](input=HepRelVertex#40,group={0, 1, 2},agg#0=sum($3)), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#40 = rel#39:HiveProject.HIVE.[](input=HepRelVertex#55,$f0=$0,$f1=$23,$f2=$26,$f3=*($5, -(CAST(1):DOUBLE, $6))), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#55 = rel#54:HiveJoin.HIVE.[](left=HepRelVertex#52,right=HepRelVertex#53,condition==($31, $20),joinType=inner), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#52 = rel#49:HiveFilter.HIVE.[](input=HepRelVertex#33,condition=AND(<($23, '1995-03-15'), >($10, '1995-03-15'))), rowcount=1.0, cumulative cost={2.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#53 = rel#50:HiveFilter.HIVE.[](input=HepRelVertex#34,condition==($6, 'BUILDING')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#33 = rel#32:HiveJoin.HIVE.[](left=HepRelVertex#30,right=HepRelVertex#31,condition==($0, $19),joinType=inner), rowcount=1.0, cumulative cost={2.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#34 = rel#2:HiveTableScan.HIVE.[](table=[default.customer]), rowcount=1.0, cumulative cost={0} HepRelVertex#30 = rel#0:HiveTableScan.HIVE.[](table=[default.lineitem]), rowcount=1.0, cumulative cost={0} HepRelVertex#31 = rel#1:HiveTableScan.HIVE.[](table=[default.orders]), rowcount=1.0, cumulative cost={0} } Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.hep.HepPlanner collectGarbage FINEST: collecting garbage Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#1: Apply rule [ReduceExpressionsRule(Join)] to [rel#54:HiveJoin.HIVE.[](left=HepRelVertex#52,right=HepRelVertex#53,condition==($31, $20),joinType=inner)] Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#2: Apply rule [PushFilterRule:HiveFilterJoinRule:no-filter] to [rel#54:HiveJoin.HIVE.[](left=HepRelVertex#52,right=HepRelVertex#53,condition==($31, $20),joinType=inner)] Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#3: Apply rule [PushFilterRule:HiveFilterJoinRule:filter] to [rel#49:HiveFilter.HIVE.[](input=HepRelVertex#33,condition=AND(<($23, '1995-03-15'), >($10, '1995-03-15'))), rel#32:HiveJoin.HIVE.[](left=HepRelVertex#30,right=HepRelVertex#31,condition==($0, $19),joinType=inner)] Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#56 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#57 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#58 Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyTransformation FINE: call#3: Rule PushFilterRule:HiveFilterJoinRule:filter arguments [rel#49:HiveFilter.HIVE.[](input=HepRelVertex#33,condition=AND(<($23, '1995-03-15'), >($10, '1995-03-15'))), rel#32:HiveJoin.HIVE.[](left=HepRelVertex#30,right=HepRelVertex#31,condition==($0, $19),joinType=inner)] produced HiveJoin#58 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#59 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#60 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#61 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#62 Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.hep.HepPlanner dumpGraph FINER: Breadth-first from root: { HepRelVertex#48 = rel#47:HiveSort.HIVE.[](input=HepRelVertex#46,fetch=10), rowcount=1.0, cumulative cost={36.0 rows, 2.0 cpu, 0.0 io} HepRelVertex#46 = rel#45:HiveSort.HIVE.[1 DESC, 2](input=HepRelVertex#44,sort0=$1,sort1=$2,dir0=DESC,dir1=ASC), rowcount=1.0, cumulative cost={20.0 rows, 1.0 cpu, 0.0 io} HepRelVertex#44 = rel#43:HiveProject.HIVE.[](input=HepRelVertex#42,l_orderkey=$0,revenue=$3,o_orderdate=$1,o_shippriority=$2), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#42 = rel#41:HiveAggregate.HIVE.[](input=HepRelVertex#40,group={0, 1, 2},agg#0=sum($3)), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#40 = rel#39:HiveProject.HIVE.[](input=HepRelVertex#55,$f0=$0,$f1=$23,$f2=$26,$f3=*($5, -(CAST(1):DOUBLE, $6))), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#55 = rel#54:HiveJoin.HIVE.[](left=HepRelVertex#62,right=HepRelVertex#53,condition==($31, $20),joinType=inner), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#53 = rel#50:HiveFilter.HIVE.[](input=HepRelVertex#34,condition==($6, 'BUILDING')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#62 = rel#61:HiveJoin.HIVE.[](left=HepRelVertex#59,right=HepRelVertex#60,condition==($0, $19),joinType=inner), rowcount=1.0, cumulative cost={2.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#34 = rel#2:HiveTableScan.HIVE.[](table=[default.customer]), rowcount=1.0, cumulative cost={0} HepRelVertex#59 = rel#56:HiveFilter.HIVE.[](input=HepRelVertex#30,condition=>($10, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#60 = rel#57:HiveFilter.HIVE.[](input=HepRelVertex#31,condition=<($4, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#30 = rel#0:HiveTableScan.HIVE.[](table=[default.lineitem]), rowcount=1.0, cumulative cost={0} HepRelVertex#31 = rel#1:HiveTableScan.HIVE.[](table=[default.orders]), rowcount=1.0, cumulative cost={0} } Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.hep.HepPlanner collectGarbage FINEST: collecting garbage Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#4: Apply rule [ReduceExpressionsRule(Join)] to [rel#54:HiveJoin.HIVE.[](left=HepRelVertex#62,right=HepRelVertex#53,condition==($31, $20),joinType=inner)] Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#5: Apply rule [PushFilterRule:HiveFilterJoinRule:no-filter] to [rel#54:HiveJoin.HIVE.[](left=HepRelVertex#62,right=HepRelVertex#53,condition==($31, $20),joinType=inner)] Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#6: Apply rule [ReduceExpressionsRule(Join)] to [rel#61:HiveJoin.HIVE.[](left=HepRelVertex#59,right=HepRelVertex#60,condition==($0, $19),joinType=inner)] Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#7: Apply rule [PushFilterRule:HiveFilterJoinRule:no-filter] to [rel#61:HiveJoin.HIVE.[](left=HepRelVertex#59,right=HepRelVertex#60,condition==($0, $19),joinType=inner)] Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#47:HiveSort.HIVE.[](input=HepRelVertex#46,fetch=10) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#45:HiveSort.HIVE.[1 DESC, 2](input=HepRelVertex#44,sort0=$1,sort1=$2,dir0=DESC,dir1=ASC) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#43:HiveProject.HIVE.[](input=HepRelVertex#42,l_orderkey=$0,revenue=$3,o_orderdate=$1,o_shippriority=$2) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#41:HiveAggregate.HIVE.[](input=HepRelVertex#40,group={0, 1, 2},agg#0=sum($3)) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#39:HiveProject.HIVE.[](input=HepRelVertex#55,$f0=$0,$f1=$23,$f2=$26,$f3=*($5, -(CAST(1):DOUBLE, $6))) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#54:HiveJoin.HIVE.[](left=HepRelVertex#62,right=HepRelVertex#53,condition==($31, $20),joinType=inner) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#61:HiveJoin.HIVE.[](left=HepRelVertex#59,right=HepRelVertex#60,condition==($0, $19),joinType=inner) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#56:HiveFilter.HIVE.[](input=HepRelVertex#30,condition=>($10, '1995-03-15')) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#0:HiveTableScan.HIVE.[](table=[default.lineitem]) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#57:HiveFilter.HIVE.[](input=HepRelVertex#31,condition=<($4, '1995-03-15')) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#1:HiveTableScan.HIVE.[](table=[default.orders]) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#50:HiveFilter.HIVE.[](input=HepRelVertex#34,condition==($6, 'BUILDING')) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#2:HiveTableScan.HIVE.[](table=[default.customer]) Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#63 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#64 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#65 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#66 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#67 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#68 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#69 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#70 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#71 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#72 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#73 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#74 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#75 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#76 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#77 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveAggregate#78 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#79 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#80 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#81 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveSort#82 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#83 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveSort#84 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#85 Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.hep.HepPlanner dumpGraph FINER: Breadth-first from root: { HepRelVertex#85 = rel#84:HiveSort.HIVE.[](input=HepRelVertex#83,fetch=10), rowcount=1.0, cumulative cost={36.0 rows, 2.0 cpu, 0.0 io} HepRelVertex#83 = rel#82:HiveSort.HIVE.[1 DESC, 2](input=HepRelVertex#81,sort0=$1,sort1=$2,dir0=DESC,dir1=ASC), rowcount=1.0, cumulative cost={20.0 rows, 1.0 cpu, 0.0 io} HepRelVertex#81 = rel#80:HiveProject.HIVE.[](input=HepRelVertex#79,l_orderkey=$0,revenue=$3,o_orderdate=$1,o_shippriority=$2), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#79 = rel#78:HiveAggregate.HIVE.[](input=HepRelVertex#77,group={0, 1, 2},agg#0=sum($3)), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#77 = rel#76:HiveProject.HIVE.[](input=HepRelVertex#75,$f0=$0,$f1=$23,$f2=$26,$f3=*($5, -(CAST(1):DOUBLE, $6))), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#75 = rel#74:HiveJoin.HIVE.[](left=HepRelVertex#70,right=HepRelVertex#73,condition==($31, $20),joinType=inner), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#70 = rel#69:HiveJoin.HIVE.[](left=HepRelVertex#65,right=HepRelVertex#68,condition==($0, $19),joinType=inner), rowcount=1.0, cumulative cost={2.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#73 = rel#72:HiveFilter.HIVE.[](input=HepRelVertex#71,condition==($6, 'BUILDING')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#65 = rel#64:HiveFilter.HIVE.[](input=HepRelVertex#63,condition=>($10, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#68 = rel#67:HiveFilter.HIVE.[](input=HepRelVertex#66,condition=<($4, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#71 = rel#2:HiveTableScan.HIVE.[](table=[default.customer]), rowcount=1.0, cumulative cost={0} HepRelVertex#63 = rel#0:HiveTableScan.HIVE.[](table=[default.lineitem]), rowcount=1.0, cumulative cost={0} HepRelVertex#66 = rel#1:HiveTableScan.HIVE.[](table=[default.orders]), rowcount=1.0, cumulative cost={0} } Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.hep.HepPlanner applyRules FINEST: Applying rule set [JoinPushTransitivePredicatesRule] Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#8: Apply rule [JoinPushTransitivePredicatesRule] to [rel#74:HiveJoin.HIVE.[](left=HepRelVertex#70,right=HepRelVertex#73,condition==($31, $20),joinType=inner)] Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#9: Apply rule [JoinPushTransitivePredicatesRule] to [rel#69:HiveJoin.HIVE.[](left=HepRelVertex#65,right=HepRelVertex#68,condition==($0, $19),joinType=inner)] Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.hep.HepPlanner applyRules FINEST: Applying rule set [HivePartitionPruneRule] Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#10: Apply rule [HivePartitionPruneRule] to [rel#64:HiveFilter.HIVE.[](input=HepRelVertex#63,condition=>($10, '1995-03-15')), rel#0:HiveTableScan.HIVE.[](table=[default.lineitem])] Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#11: Apply rule [HivePartitionPruneRule] to [rel#67:HiveFilter.HIVE.[](input=HepRelVertex#66,condition=<($4, '1995-03-15')), rel#1:HiveTableScan.HIVE.[](table=[default.orders])] Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#12: Apply rule [HivePartitionPruneRule] to [rel#72:HiveFilter.HIVE.[](input=HepRelVertex#71,condition==($6, 'BUILDING')), rel#2:HiveTableScan.HIVE.[](table=[default.customer])] Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#84:HiveSort.HIVE.[](input=HepRelVertex#83,fetch=10) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#82:HiveSort.HIVE.[1 DESC, 2](input=HepRelVertex#81,sort0=$1,sort1=$2,dir0=DESC,dir1=ASC) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#80:HiveProject.HIVE.[](input=HepRelVertex#79,l_orderkey=$0,revenue=$3,o_orderdate=$1,o_shippriority=$2) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#78:HiveAggregate.HIVE.[](input=HepRelVertex#77,group={0, 1, 2},agg#0=sum($3)) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#76:HiveProject.HIVE.[](input=HepRelVertex#75,$f0=$0,$f1=$23,$f2=$26,$f3=*($5, -(CAST(1):DOUBLE, $6))) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#74:HiveJoin.HIVE.[](left=HepRelVertex#70,right=HepRelVertex#73,condition==($31, $20),joinType=inner) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#69:HiveJoin.HIVE.[](left=HepRelVertex#65,right=HepRelVertex#68,condition==($0, $19),joinType=inner) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#64:HiveFilter.HIVE.[](input=HepRelVertex#63,condition=>($10, '1995-03-15')) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#0:HiveTableScan.HIVE.[](table=[default.lineitem]) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#67:HiveFilter.HIVE.[](input=HepRelVertex#66,condition=<($4, '1995-03-15')) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#1:HiveTableScan.HIVE.[](table=[default.orders]) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#72:HiveFilter.HIVE.[](input=HepRelVertex#71,condition==($6, 'BUILDING')) Jan 29, 2015 11:48:03 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#2:HiveTableScan.HIVE.[](table=[default.customer]) Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#86 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#87 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#88 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#89 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#90 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#91 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#92 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#93 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#94 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveAggregate#95 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#96 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveSort#97 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveSort#98 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#99 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#100 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#101 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#102 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#103 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#104 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#105 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#106 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#107 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#108 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#109 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#110 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#111 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#112 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#113 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#114 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#115 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#116 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#117 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#118 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#119 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveAggregate#120 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#121 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#122 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#123 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveSort#124 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#125 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveSort#126 Jan 29, 2015 11:48:03 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#127 15/01/29 11:48:03 [main]: INFO metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=lineitem 15/01/29 11:48:03 [main]: INFO HiveMetaStore.audit: ugi=zhangyun ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=lineitem 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.start(ObjectStore.java:2388) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 2, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:972) 15/01/29 11:48:03 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 1, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:980) 15/01/29 11:48:04 [main]: DEBUG metastore.MetaStoreDirectSql: Direct SQL query in 9.369262ms + 3.13558ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (?,?,?)] 15/01/29 11:48:04 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 0, isactive true at: org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.commit(ObjectStore.java:2425) 15/01/29 11:48:04 [main]: DEBUG metastore.ObjectStore: statistics for 3 columns retrieved using SQL in 34.258435ms 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Identified Primary - Foreign Key relation: 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: HiveJoin(condition=[=($0, $4)], joinType=[inner]) HiveFilter(condition=[>($3, '1995-03-15')]) HiveProject(l_orderkey=[$0], l_extendedprice=[$5], l_discount=[$6], l_shipdate=[$10]) HiveTableScan(table=[[default.lineitem]]) HiveFilter(condition=[<($2, '1995-03-15')]) HiveProject(o_orderkey=[$0], o_custkey=[$1], o_orderdate=[$4], o_shippriority=[$7]) HiveTableScan(table=[[default.orders]]) 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Primary - Foreign Key join: fkSide = 1 FKInfo:FKInfo(rowCount=1.00,ndv=1.00) PKInfo:PKInfo(rowCount=1.00,ndv=1.00,selectivity=1.00) isPKSideSimple:true NDV Scaling Factor:1.00 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Identified Primary - Foreign Key relation: 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: HiveJoin(condition=[=($8, $5)], joinType=[inner]) HiveJoin(condition=[=($0, $4)], joinType=[inner]) HiveFilter(condition=[>($3, '1995-03-15')]) HiveProject(l_orderkey=[$0], l_extendedprice=[$5], l_discount=[$6], l_shipdate=[$10]) HiveTableScan(table=[[default.lineitem]]) HiveFilter(condition=[<($2, '1995-03-15')]) HiveProject(o_orderkey=[$0], o_custkey=[$1], o_orderdate=[$4], o_shippriority=[$7]) HiveTableScan(table=[[default.orders]]) HiveFilter(condition=[=($1, 'BUILDING')]) HiveProject(c_custkey=[$0], c_mktsegment=[$6]) HiveTableScan(table=[[default.customer]]) 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Primary - Foreign Key join: fkSide = 1 FKInfo:FKInfo(rowCount=1.00,ndv=1.00) PKInfo:PKInfo(rowCount=1.00,ndv=1.00,selectivity=1.00) isPKSideSimple:true NDV Scaling Factor:1.00 Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.hep.HepPlanner dumpGraph FINER: Breadth-first from root: { HepRelVertex#127 = rel#126:HiveSort.HIVE.[](input=HepRelVertex#125,fetch=10), rowcount=1.0, cumulative cost={36.0 rows, 2.0 cpu, 0.0 io} HepRelVertex#125 = rel#124:HiveSort.HIVE.[1 DESC, 2](input=HepRelVertex#123,sort0=$1,sort1=$2,dir0=DESC,dir1=ASC), rowcount=1.0, cumulative cost={20.0 rows, 1.0 cpu, 0.0 io} HepRelVertex#123 = rel#122:HiveProject.HIVE.[](input=HepRelVertex#121,l_orderkey=$0,revenue=$3,o_orderdate=$1,o_shippriority=$2), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#121 = rel#120:HiveAggregate.HIVE.[](input=HepRelVertex#119,group={0, 1, 2},agg#0=sum($3)), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#119 = rel#118:HiveProject.HIVE.[](input=HepRelVertex#117,$f0=$0,$f1=$6,$f2=$7,$f3=*($1, -(CAST(1):DOUBLE, $2))), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#117 = rel#116:HiveJoin.HIVE.[](left=HepRelVertex#110,right=HepRelVertex#115,condition==($8, $5),joinType=inner), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#110 = rel#109:HiveJoin.HIVE.[](left=HepRelVertex#103,right=HepRelVertex#108,condition==($0, $4),joinType=inner), rowcount=1.0, cumulative cost={2.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#115 = rel#114:HiveFilter.HIVE.[](input=HepRelVertex#113,condition==($1, 'BUILDING')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#103 = rel#102:HiveFilter.HIVE.[](input=HepRelVertex#101,condition=>($3, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#108 = rel#107:HiveFilter.HIVE.[](input=HepRelVertex#106,condition=<($2, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#113 = rel#112:HiveProject.HIVE.[](input=HepRelVertex#111,c_custkey=$0,c_mktsegment=$6), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#101 = rel#100:HiveProject.HIVE.[](input=HepRelVertex#99,l_orderkey=$0,l_extendedprice=$5,l_discount=$6,l_shipdate=$10), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#106 = rel#105:HiveProject.HIVE.[](input=HepRelVertex#104,o_orderkey=$0,o_custkey=$1,o_orderdate=$4,o_shippriority=$7), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#111 = rel#2:HiveTableScan.HIVE.[](table=[default.customer]), rowcount=1.0, cumulative cost={0} HepRelVertex#99 = rel#0:HiveTableScan.HIVE.[](table=[default.lineitem]), rowcount=1.0, cumulative cost={0} HepRelVertex#104 = rel#1:HiveTableScan.HIVE.[](table=[default.orders]), rowcount=1.0, cumulative cost={0} } Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.hep.HepPlanner executeInstruction FINEST: Setting match limit to TOP_DOWN Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.hep.HepPlanner applyRules FINEST: Applying rule set [FilterProjectTransposeRule] Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#13: Apply rule [FilterProjectTransposeRule] to [rel#114:HiveFilter.HIVE.[](input=HepRelVertex#113,condition==($1, 'BUILDING')), rel#112:HiveProject.HIVE.[](input=HepRelVertex#111,c_custkey=$0,c_mktsegment=$6)] Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#128 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#129 Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyTransformation FINE: call#13: Rule FilterProjectTransposeRule arguments [rel#114:HiveFilter.HIVE.[](input=HepRelVertex#113,condition==($1, 'BUILDING')), rel#112:HiveProject.HIVE.[](input=HepRelVertex#111,c_custkey=$0,c_mktsegment=$6)] produced HiveProject#129 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#130 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#131 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#132 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Identified Primary - Foreign Key relation: 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: HiveJoin(condition=[=($0, $4)], joinType=[inner]) HiveFilter(condition=[>($3, '1995-03-15')]) HiveProject(l_orderkey=[$0], l_extendedprice=[$5], l_discount=[$6], l_shipdate=[$10]) HiveTableScan(table=[[default.lineitem]]) HiveFilter(condition=[<($2, '1995-03-15')]) HiveProject(o_orderkey=[$0], o_custkey=[$1], o_orderdate=[$4], o_shippriority=[$7]) HiveTableScan(table=[[default.orders]]) 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Primary - Foreign Key join: fkSide = 1 FKInfo:FKInfo(rowCount=1.00,ndv=1.00) PKInfo:PKInfo(rowCount=1.00,ndv=1.00,selectivity=1.00) isPKSideSimple:true NDV Scaling Factor:1.00 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Identified Primary - Foreign Key relation: 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: HiveJoin(condition=[=($8, $5)], joinType=[inner]) HiveJoin(condition=[=($0, $4)], joinType=[inner]) HiveFilter(condition=[>($3, '1995-03-15')]) HiveProject(l_orderkey=[$0], l_extendedprice=[$5], l_discount=[$6], l_shipdate=[$10]) HiveTableScan(table=[[default.lineitem]]) HiveFilter(condition=[<($2, '1995-03-15')]) HiveProject(o_orderkey=[$0], o_custkey=[$1], o_orderdate=[$4], o_shippriority=[$7]) HiveTableScan(table=[[default.orders]]) HiveProject(c_custkey=[$0], c_mktsegment=[$6]) HiveFilter(condition=[=($6, 'BUILDING')]) HiveTableScan(table=[[default.customer]]) 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Primary - Foreign Key join: fkSide = 1 FKInfo:FKInfo(rowCount=1.00,ndv=-1.00) PKInfo:PKInfo(rowCount=1.00,ndv=-1.00,selectivity=1.00) isPKSideSimple:false NDV Scaling Factor:1.00 Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.hep.HepPlanner dumpGraph FINER: Breadth-first from root: { HepRelVertex#127 = rel#126:HiveSort.HIVE.[](input=HepRelVertex#125,fetch=10), rowcount=1.0, cumulative cost={36.0 rows, 2.0 cpu, 0.0 io} HepRelVertex#125 = rel#124:HiveSort.HIVE.[1 DESC, 2](input=HepRelVertex#123,sort0=$1,sort1=$2,dir0=DESC,dir1=ASC), rowcount=1.0, cumulative cost={20.0 rows, 1.0 cpu, 0.0 io} HepRelVertex#123 = rel#122:HiveProject.HIVE.[](input=HepRelVertex#121,l_orderkey=$0,revenue=$3,o_orderdate=$1,o_shippriority=$2), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#121 = rel#120:HiveAggregate.HIVE.[](input=HepRelVertex#119,group={0, 1, 2},agg#0=sum($3)), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#119 = rel#118:HiveProject.HIVE.[](input=HepRelVertex#117,$f0=$0,$f1=$6,$f2=$7,$f3=*($1, -(CAST(1):DOUBLE, $2))), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#117 = rel#116:HiveJoin.HIVE.[](left=HepRelVertex#110,right=HepRelVertex#132,condition==($8, $5),joinType=inner), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#110 = rel#109:HiveJoin.HIVE.[](left=HepRelVertex#103,right=HepRelVertex#108,condition==($0, $4),joinType=inner), rowcount=1.0, cumulative cost={2.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#132 = rel#131:HiveProject.HIVE.[](input=HepRelVertex#130,c_custkey=$0,c_mktsegment=$6), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#103 = rel#102:HiveFilter.HIVE.[](input=HepRelVertex#101,condition=>($3, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#108 = rel#107:HiveFilter.HIVE.[](input=HepRelVertex#106,condition=<($2, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#130 = rel#128:HiveFilter.HIVE.[](input=HepRelVertex#111,condition==($6, 'BUILDING')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#101 = rel#100:HiveProject.HIVE.[](input=HepRelVertex#99,l_orderkey=$0,l_extendedprice=$5,l_discount=$6,l_shipdate=$10), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#106 = rel#105:HiveProject.HIVE.[](input=HepRelVertex#104,o_orderkey=$0,o_custkey=$1,o_orderdate=$4,o_shippriority=$7), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#111 = rel#2:HiveTableScan.HIVE.[](table=[default.customer]), rowcount=1.0, cumulative cost={0} HepRelVertex#99 = rel#0:HiveTableScan.HIVE.[](table=[default.lineitem]), rowcount=1.0, cumulative cost={0} HepRelVertex#104 = rel#1:HiveTableScan.HIVE.[](table=[default.orders]), rowcount=1.0, cumulative cost={0} } Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.hep.HepPlanner collectGarbage FINEST: collecting garbage Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#14: Apply rule [FilterProjectTransposeRule] to [rel#102:HiveFilter.HIVE.[](input=HepRelVertex#101,condition=>($3, '1995-03-15')), rel#100:HiveProject.HIVE.[](input=HepRelVertex#99,l_orderkey=$0,l_extendedprice=$5,l_discount=$6,l_shipdate=$10)] Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#133 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#134 Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyTransformation FINE: call#14: Rule FilterProjectTransposeRule arguments [rel#102:HiveFilter.HIVE.[](input=HepRelVertex#101,condition=>($3, '1995-03-15')), rel#100:HiveProject.HIVE.[](input=HepRelVertex#99,l_orderkey=$0,l_extendedprice=$5,l_discount=$6,l_shipdate=$10)] produced HiveProject#134 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#135 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#136 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#137 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Identified Primary - Foreign Key relation: 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: HiveJoin(condition=[=($0, $4)], joinType=[inner]) HiveProject(l_orderkey=[$0], l_extendedprice=[$5], l_discount=[$6], l_shipdate=[$10]) HiveFilter(condition=[>($10, '1995-03-15')]) HiveTableScan(table=[[default.lineitem]]) HiveFilter(condition=[<($2, '1995-03-15')]) HiveProject(o_orderkey=[$0], o_custkey=[$1], o_orderdate=[$4], o_shippriority=[$7]) HiveTableScan(table=[[default.orders]]) 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Primary - Foreign Key join: fkSide = 1 FKInfo:FKInfo(rowCount=1.00,ndv=-1.00) PKInfo:PKInfo(rowCount=1.00,ndv=-1.00,selectivity=1.00) isPKSideSimple:false NDV Scaling Factor:1.00 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Identified Primary - Foreign Key relation: 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: HiveJoin(condition=[=($8, $5)], joinType=[inner]) HiveJoin(condition=[=($0, $4)], joinType=[inner]) HiveProject(l_orderkey=[$0], l_extendedprice=[$5], l_discount=[$6], l_shipdate=[$10]) HiveFilter(condition=[>($10, '1995-03-15')]) HiveTableScan(table=[[default.lineitem]]) HiveFilter(condition=[<($2, '1995-03-15')]) HiveProject(o_orderkey=[$0], o_custkey=[$1], o_orderdate=[$4], o_shippriority=[$7]) HiveTableScan(table=[[default.orders]]) HiveProject(c_custkey=[$0], c_mktsegment=[$6]) HiveFilter(condition=[=($6, 'BUILDING')]) HiveTableScan(table=[[default.customer]]) 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Primary - Foreign Key join: fkSide = 1 FKInfo:FKInfo(rowCount=1.00,ndv=-1.00) PKInfo:PKInfo(rowCount=1.00,ndv=-1.00,selectivity=1.00) isPKSideSimple:false NDV Scaling Factor:1.00 Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.hep.HepPlanner dumpGraph FINER: Breadth-first from root: { HepRelVertex#127 = rel#126:HiveSort.HIVE.[](input=HepRelVertex#125,fetch=10), rowcount=1.0, cumulative cost={36.0 rows, 2.0 cpu, 0.0 io} HepRelVertex#125 = rel#124:HiveSort.HIVE.[1 DESC, 2](input=HepRelVertex#123,sort0=$1,sort1=$2,dir0=DESC,dir1=ASC), rowcount=1.0, cumulative cost={20.0 rows, 1.0 cpu, 0.0 io} HepRelVertex#123 = rel#122:HiveProject.HIVE.[](input=HepRelVertex#121,l_orderkey=$0,revenue=$3,o_orderdate=$1,o_shippriority=$2), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#121 = rel#120:HiveAggregate.HIVE.[](input=HepRelVertex#119,group={0, 1, 2},agg#0=sum($3)), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#119 = rel#118:HiveProject.HIVE.[](input=HepRelVertex#117,$f0=$0,$f1=$6,$f2=$7,$f3=*($1, -(CAST(1):DOUBLE, $2))), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#117 = rel#116:HiveJoin.HIVE.[](left=HepRelVertex#110,right=HepRelVertex#132,condition==($8, $5),joinType=inner), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#110 = rel#109:HiveJoin.HIVE.[](left=HepRelVertex#137,right=HepRelVertex#108,condition==($0, $4),joinType=inner), rowcount=1.0, cumulative cost={2.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#132 = rel#131:HiveProject.HIVE.[](input=HepRelVertex#130,c_custkey=$0,c_mktsegment=$6), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#108 = rel#107:HiveFilter.HIVE.[](input=HepRelVertex#106,condition=<($2, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#137 = rel#136:HiveProject.HIVE.[](input=HepRelVertex#135,l_orderkey=$0,l_extendedprice=$5,l_discount=$6,l_shipdate=$10), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#130 = rel#128:HiveFilter.HIVE.[](input=HepRelVertex#111,condition==($6, 'BUILDING')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#106 = rel#105:HiveProject.HIVE.[](input=HepRelVertex#104,o_orderkey=$0,o_custkey=$1,o_orderdate=$4,o_shippriority=$7), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#135 = rel#133:HiveFilter.HIVE.[](input=HepRelVertex#99,condition=>($10, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#111 = rel#2:HiveTableScan.HIVE.[](table=[default.customer]), rowcount=1.0, cumulative cost={0} HepRelVertex#104 = rel#1:HiveTableScan.HIVE.[](table=[default.orders]), rowcount=1.0, cumulative cost={0} HepRelVertex#99 = rel#0:HiveTableScan.HIVE.[](table=[default.lineitem]), rowcount=1.0, cumulative cost={0} } Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.hep.HepPlanner collectGarbage FINEST: collecting garbage Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#15: Apply rule [FilterProjectTransposeRule] to [rel#107:HiveFilter.HIVE.[](input=HepRelVertex#106,condition=<($2, '1995-03-15')), rel#105:HiveProject.HIVE.[](input=HepRelVertex#104,o_orderkey=$0,o_custkey=$1,o_orderdate=$4,o_shippriority=$7)] Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#138 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#139 Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyTransformation FINE: call#15: Rule FilterProjectTransposeRule arguments [rel#107:HiveFilter.HIVE.[](input=HepRelVertex#106,condition=<($2, '1995-03-15')), rel#105:HiveProject.HIVE.[](input=HepRelVertex#104,o_orderkey=$0,o_custkey=$1,o_orderdate=$4,o_shippriority=$7)] produced HiveProject#139 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#140 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#141 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#142 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Identified Primary - Foreign Key relation: 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: HiveJoin(condition=[=($0, $4)], joinType=[inner]) HiveProject(l_orderkey=[$0], l_extendedprice=[$5], l_discount=[$6], l_shipdate=[$10]) HiveFilter(condition=[>($10, '1995-03-15')]) HiveTableScan(table=[[default.lineitem]]) HiveProject(o_orderkey=[$0], o_custkey=[$1], o_orderdate=[$4], o_shippriority=[$7]) HiveFilter(condition=[<($4, '1995-03-15')]) HiveTableScan(table=[[default.orders]]) 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Primary - Foreign Key join: fkSide = 1 FKInfo:FKInfo(rowCount=1.00,ndv=-1.00) PKInfo:PKInfo(rowCount=1.00,ndv=-1.00,selectivity=1.00) isPKSideSimple:false NDV Scaling Factor:1.00 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Identified Primary - Foreign Key relation: 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: HiveJoin(condition=[=($8, $5)], joinType=[inner]) HiveJoin(condition=[=($0, $4)], joinType=[inner]) HiveProject(l_orderkey=[$0], l_extendedprice=[$5], l_discount=[$6], l_shipdate=[$10]) HiveFilter(condition=[>($10, '1995-03-15')]) HiveTableScan(table=[[default.lineitem]]) HiveProject(o_orderkey=[$0], o_custkey=[$1], o_orderdate=[$4], o_shippriority=[$7]) HiveFilter(condition=[<($4, '1995-03-15')]) HiveTableScan(table=[[default.orders]]) HiveProject(c_custkey=[$0], c_mktsegment=[$6]) HiveFilter(condition=[=($6, 'BUILDING')]) HiveTableScan(table=[[default.customer]]) 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Primary - Foreign Key join: fkSide = 1 FKInfo:FKInfo(rowCount=1.00,ndv=-1.00) PKInfo:PKInfo(rowCount=1.00,ndv=-1.00,selectivity=1.00) isPKSideSimple:false NDV Scaling Factor:1.00 Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.hep.HepPlanner dumpGraph FINER: Breadth-first from root: { HepRelVertex#127 = rel#126:HiveSort.HIVE.[](input=HepRelVertex#125,fetch=10), rowcount=1.0, cumulative cost={36.0 rows, 2.0 cpu, 0.0 io} HepRelVertex#125 = rel#124:HiveSort.HIVE.[1 DESC, 2](input=HepRelVertex#123,sort0=$1,sort1=$2,dir0=DESC,dir1=ASC), rowcount=1.0, cumulative cost={20.0 rows, 1.0 cpu, 0.0 io} HepRelVertex#123 = rel#122:HiveProject.HIVE.[](input=HepRelVertex#121,l_orderkey=$0,revenue=$3,o_orderdate=$1,o_shippriority=$2), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#121 = rel#120:HiveAggregate.HIVE.[](input=HepRelVertex#119,group={0, 1, 2},agg#0=sum($3)), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#119 = rel#118:HiveProject.HIVE.[](input=HepRelVertex#117,$f0=$0,$f1=$6,$f2=$7,$f3=*($1, -(CAST(1):DOUBLE, $2))), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#117 = rel#116:HiveJoin.HIVE.[](left=HepRelVertex#110,right=HepRelVertex#132,condition==($8, $5),joinType=inner), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#110 = rel#109:HiveJoin.HIVE.[](left=HepRelVertex#137,right=HepRelVertex#142,condition==($0, $4),joinType=inner), rowcount=1.0, cumulative cost={2.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#132 = rel#131:HiveProject.HIVE.[](input=HepRelVertex#130,c_custkey=$0,c_mktsegment=$6), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#137 = rel#136:HiveProject.HIVE.[](input=HepRelVertex#135,l_orderkey=$0,l_extendedprice=$5,l_discount=$6,l_shipdate=$10), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#142 = rel#141:HiveProject.HIVE.[](input=HepRelVertex#140,o_orderkey=$0,o_custkey=$1,o_orderdate=$4,o_shippriority=$7), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#130 = rel#128:HiveFilter.HIVE.[](input=HepRelVertex#111,condition==($6, 'BUILDING')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#135 = rel#133:HiveFilter.HIVE.[](input=HepRelVertex#99,condition=>($10, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#140 = rel#138:HiveFilter.HIVE.[](input=HepRelVertex#104,condition=<($4, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#111 = rel#2:HiveTableScan.HIVE.[](table=[default.customer]), rowcount=1.0, cumulative cost={0} HepRelVertex#99 = rel#0:HiveTableScan.HIVE.[](table=[default.lineitem]), rowcount=1.0, cumulative cost={0} HepRelVertex#104 = rel#1:HiveTableScan.HIVE.[](table=[default.orders]), rowcount=1.0, cumulative cost={0} } Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.hep.HepPlanner collectGarbage FINEST: collecting garbage Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#126:HiveSort.HIVE.[](input=HepRelVertex#125,fetch=10) Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#124:HiveSort.HIVE.[1 DESC, 2](input=HepRelVertex#123,sort0=$1,sort1=$2,dir0=DESC,dir1=ASC) Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#122:HiveProject.HIVE.[](input=HepRelVertex#121,l_orderkey=$0,revenue=$3,o_orderdate=$1,o_shippriority=$2) Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#120:HiveAggregate.HIVE.[](input=HepRelVertex#119,group={0, 1, 2},agg#0=sum($3)) Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#118:HiveProject.HIVE.[](input=HepRelVertex#117,$f0=$0,$f1=$6,$f2=$7,$f3=*($1, -(CAST(1):DOUBLE, $2))) Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#116:HiveJoin.HIVE.[](left=HepRelVertex#110,right=HepRelVertex#132,condition==($8, $5),joinType=inner) Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#109:HiveJoin.HIVE.[](left=HepRelVertex#137,right=HepRelVertex#142,condition==($0, $4),joinType=inner) Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#136:HiveProject.HIVE.[](input=HepRelVertex#135,l_orderkey=$0,l_extendedprice=$5,l_discount=$6,l_shipdate=$10) Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#133:HiveFilter.HIVE.[](input=HepRelVertex#99,condition=>($10, '1995-03-15')) Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#0:HiveTableScan.HIVE.[](table=[default.lineitem]) Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#141:HiveProject.HIVE.[](input=HepRelVertex#140,o_orderkey=$0,o_custkey=$1,o_orderdate=$4,o_shippriority=$7) Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#138:HiveFilter.HIVE.[](input=HepRelVertex#104,condition=<($4, '1995-03-15')) Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#1:HiveTableScan.HIVE.[](table=[default.orders]) Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#131:HiveProject.HIVE.[](input=HepRelVertex#130,c_custkey=$0,c_mktsegment=$6) Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#128:HiveFilter.HIVE.[](input=HepRelVertex#111,condition==($6, 'BUILDING')) Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyChosen FINE: For final plan, using rel#2:HiveTableScan.HIVE.[](table=[default.customer]) >>>>>>>>>>>>>>>> Apply Pre Join Order optimizations finished! Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#143 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#144 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#145 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#146 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#147 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#148 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#149 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#150 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#151 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#152 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#153 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#154 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#155 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#156 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#157 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#158 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#159 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#160 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#161 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#162 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#163 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveAggregate#164 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#165 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#166 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#167 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveSort#168 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#169 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveSort#170 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#171 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Identified Primary - Foreign Key relation: 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: HiveJoin(condition=[=($0, $4)], joinType=[inner]) HiveProject(l_orderkey=[$0], l_extendedprice=[$5], l_discount=[$6], l_shipdate=[$10]) HiveFilter(condition=[>($10, '1995-03-15')]) HiveTableScan(table=[[default.lineitem]]) HiveProject(o_orderkey=[$0], o_custkey=[$1], o_orderdate=[$4], o_shippriority=[$7]) HiveFilter(condition=[<($4, '1995-03-15')]) HiveTableScan(table=[[default.orders]]) 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Primary - Foreign Key join: fkSide = 1 FKInfo:FKInfo(rowCount=1.00,ndv=-1.00) PKInfo:PKInfo(rowCount=1.00,ndv=-1.00,selectivity=1.00) isPKSideSimple:false NDV Scaling Factor:1.00 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Identified Primary - Foreign Key relation: 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: HiveJoin(condition=[=($8, $5)], joinType=[inner]) HiveJoin(condition=[=($0, $4)], joinType=[inner]) HiveProject(l_orderkey=[$0], l_extendedprice=[$5], l_discount=[$6], l_shipdate=[$10]) HiveFilter(condition=[>($10, '1995-03-15')]) HiveTableScan(table=[[default.lineitem]]) HiveProject(o_orderkey=[$0], o_custkey=[$1], o_orderdate=[$4], o_shippriority=[$7]) HiveFilter(condition=[<($4, '1995-03-15')]) HiveTableScan(table=[[default.orders]]) HiveProject(c_custkey=[$0], c_mktsegment=[$6]) HiveFilter(condition=[=($6, 'BUILDING')]) HiveTableScan(table=[[default.customer]]) 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Primary - Foreign Key join: fkSide = 1 FKInfo:FKInfo(rowCount=1.00,ndv=-1.00) PKInfo:PKInfo(rowCount=1.00,ndv=-1.00,selectivity=1.00) isPKSideSimple:false NDV Scaling Factor:1.00 Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.hep.HepPlanner dumpGraph FINER: Breadth-first from root: { HepRelVertex#171 = rel#170:HiveSort.HIVE.[](input=HepRelVertex#169,fetch=10), rowcount=1.0, cumulative cost={36.0 rows, 2.0 cpu, 0.0 io} HepRelVertex#169 = rel#168:HiveSort.HIVE.[1 DESC, 2](input=HepRelVertex#167,sort0=$1,sort1=$2,dir0=DESC,dir1=ASC), rowcount=1.0, cumulative cost={20.0 rows, 1.0 cpu, 0.0 io} HepRelVertex#167 = rel#166:HiveProject.HIVE.[](input=HepRelVertex#165,l_orderkey=$0,revenue=$3,o_orderdate=$1,o_shippriority=$2), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#165 = rel#164:HiveAggregate.HIVE.[](input=HepRelVertex#163,group={0, 1, 2},agg#0=sum($3)), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#163 = rel#162:HiveProject.HIVE.[](input=HepRelVertex#161,$f0=$0,$f1=$6,$f2=$7,$f3=*($1, -(CAST(1):DOUBLE, $2))), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#161 = rel#160:HiveJoin.HIVE.[](left=HepRelVertex#154,right=HepRelVertex#159,condition==($8, $5),joinType=inner), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#154 = rel#153:HiveJoin.HIVE.[](left=HepRelVertex#147,right=HepRelVertex#152,condition==($0, $4),joinType=inner), rowcount=1.0, cumulative cost={2.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#159 = rel#158:HiveProject.HIVE.[](input=HepRelVertex#157,c_custkey=$0,c_mktsegment=$6), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#147 = rel#146:HiveProject.HIVE.[](input=HepRelVertex#145,l_orderkey=$0,l_extendedprice=$5,l_discount=$6,l_shipdate=$10), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#152 = rel#151:HiveProject.HIVE.[](input=HepRelVertex#150,o_orderkey=$0,o_custkey=$1,o_orderdate=$4,o_shippriority=$7), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#157 = rel#156:HiveFilter.HIVE.[](input=HepRelVertex#155,condition==($6, 'BUILDING')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#145 = rel#144:HiveFilter.HIVE.[](input=HepRelVertex#143,condition=>($10, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#150 = rel#149:HiveFilter.HIVE.[](input=HepRelVertex#148,condition=<($4, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#155 = rel#2:HiveTableScan.HIVE.[](table=[default.customer]), rowcount=1.0, cumulative cost={0} HepRelVertex#143 = rel#0:HiveTableScan.HIVE.[](table=[default.lineitem]), rowcount=1.0, cumulative cost={0} HepRelVertex#148 = rel#1:HiveTableScan.HIVE.[](table=[default.orders]), rowcount=1.0, cumulative cost={0} } Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#172 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#173 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#174 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#175 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#176 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#177 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#178 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#179 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#180 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#181 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveFilter#182 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#183 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#184 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#185 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveJoin#186 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#187 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#188 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#189 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveAggregate#190 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#191 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveProject#192 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#193 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveSort#194 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#195 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HiveSort#196 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#197 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Identified Primary - Foreign Key relation: 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: HiveJoin(condition=[=($0, $4)], joinType=[inner]) HiveProject(l_orderkey=[$0], l_extendedprice=[$5], l_discount=[$6], l_shipdate=[$10]) HiveFilter(condition=[>($10, '1995-03-15')]) HiveTableScan(table=[[default.lineitem]]) HiveProject(o_orderkey=[$0], o_custkey=[$1], o_orderdate=[$4], o_shippriority=[$7]) HiveFilter(condition=[<($4, '1995-03-15')]) HiveTableScan(table=[[default.orders]]) 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Primary - Foreign Key join: fkSide = 1 FKInfo:FKInfo(rowCount=1.00,ndv=-1.00) PKInfo:PKInfo(rowCount=1.00,ndv=-1.00,selectivity=1.00) isPKSideSimple:false NDV Scaling Factor:1.00 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Identified Primary - Foreign Key relation: 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: HiveJoin(condition=[=($8, $5)], joinType=[inner]) HiveJoin(condition=[=($0, $4)], joinType=[inner]) HiveProject(l_orderkey=[$0], l_extendedprice=[$5], l_discount=[$6], l_shipdate=[$10]) HiveFilter(condition=[>($10, '1995-03-15')]) HiveTableScan(table=[[default.lineitem]]) HiveProject(o_orderkey=[$0], o_custkey=[$1], o_orderdate=[$4], o_shippriority=[$7]) HiveFilter(condition=[<($4, '1995-03-15')]) HiveTableScan(table=[[default.orders]]) HiveProject(c_custkey=[$0], c_mktsegment=[$6]) HiveFilter(condition=[=($6, 'BUILDING')]) HiveTableScan(table=[[default.customer]]) 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Primary - Foreign Key join: fkSide = 1 FKInfo:FKInfo(rowCount=1.00,ndv=-1.00) PKInfo:PKInfo(rowCount=1.00,ndv=-1.00,selectivity=1.00) isPKSideSimple:false NDV Scaling Factor:1.00 Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.hep.HepPlanner dumpGraph FINER: Breadth-first from root: { HepRelVertex#197 = rel#196:HiveSort.HIVE.[](input=HepRelVertex#195,fetch=10), rowcount=1.0, cumulative cost={36.0 rows, 2.0 cpu, 0.0 io} HepRelVertex#195 = rel#194:HiveSort.HIVE.[1 DESC, 2](input=HepRelVertex#193,sort0=$1,sort1=$2,dir0=DESC,dir1=ASC), rowcount=1.0, cumulative cost={20.0 rows, 1.0 cpu, 0.0 io} HepRelVertex#193 = rel#192:HiveProject.HIVE.[](input=HepRelVertex#191,l_orderkey=$0,revenue=$3,o_orderdate=$1,o_shippriority=$2), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#191 = rel#190:HiveAggregate.HIVE.[](input=HepRelVertex#189,group={0, 1, 2},agg#0=sum($3)), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#189 = rel#188:HiveProject.HIVE.[](input=HepRelVertex#187,$f0=$0,$f1=$6,$f2=$7,$f3=*($1, -(CAST(1):DOUBLE, $2))), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#187 = rel#186:HiveJoin.HIVE.[](left=HepRelVertex#181,right=HepRelVertex#185,condition==($8, $5),joinType=inner), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#181 = rel#180:HiveJoin.HIVE.[](left=HepRelVertex#175,right=HepRelVertex#179,condition==($0, $4),joinType=inner), rowcount=1.0, cumulative cost={2.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#185 = rel#184:HiveProject.HIVE.[](input=HepRelVertex#183,c_custkey=$0,c_mktsegment=$6), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#175 = rel#174:HiveProject.HIVE.[](input=HepRelVertex#173,l_orderkey=$0,l_extendedprice=$5,l_discount=$6,l_shipdate=$10), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#179 = rel#178:HiveProject.HIVE.[](input=HepRelVertex#177,o_orderkey=$0,o_custkey=$1,o_orderdate=$4,o_shippriority=$7), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#183 = rel#182:HiveFilter.HIVE.[](input=HepRelVertex#155,condition==($6, 'BUILDING')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#173 = rel#172:HiveFilter.HIVE.[](input=HepRelVertex#143,condition=>($10, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#177 = rel#176:HiveFilter.HIVE.[](input=HepRelVertex#148,condition=<($4, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#155 = rel#2:HiveTableScan.HIVE.[](table=[default.customer]), rowcount=1.0, cumulative cost={0} HepRelVertex#143 = rel#0:HiveTableScan.HIVE.[](table=[default.lineitem]), rowcount=1.0, cumulative cost={0} HepRelVertex#148 = rel#1:HiveTableScan.HIVE.[](table=[default.orders]), rowcount=1.0, cumulative cost={0} } Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.hep.HepPlanner executeInstruction FINEST: Setting match limit to BOTTOM_UP Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.hep.HepPlanner applyRules FINEST: Applying rule set [JoinToMultiJoinRule] Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#16: Apply rule [JoinToMultiJoinRule] to [rel#153:HiveJoin.HIVE.[](left=HepRelVertex#147,right=HepRelVertex#152,condition==($0, $4),joinType=inner), rel#146:HiveProject.HIVE.[](input=HepRelVertex#145,l_orderkey=$0,l_extendedprice=$5,l_discount=$6,l_shipdate=$10), rel#151:HiveProject.HIVE.[](input=HepRelVertex#150,o_orderkey=$0,o_custkey=$1,o_orderdate=$4,o_shippriority=$7)] Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new MultiJoin#198 Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyTransformation FINE: call#16: Rule JoinToMultiJoinRule arguments [rel#153:HiveJoin.HIVE.[](left=HepRelVertex#147,right=HepRelVertex#152,condition==($0, $4),joinType=inner), rel#146:HiveProject.HIVE.[](input=HepRelVertex#145,l_orderkey=$0,l_extendedprice=$5,l_discount=$6,l_shipdate=$10), rel#151:HiveProject.HIVE.[](input=HepRelVertex#150,o_orderkey=$0,o_custkey=$1,o_orderdate=$4,o_shippriority=$7)] produced MultiJoin#198 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new MultiJoin#199 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#200 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Identified Primary - Foreign Key relation: 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: HiveJoin(condition=[=($0, $4)], joinType=[inner]) HiveProject(l_orderkey=[$0], l_extendedprice=[$5], l_discount=[$6], l_shipdate=[$10]) HiveFilter(condition=[>($10, '1995-03-15')]) HiveTableScan(table=[[default.lineitem]]) HiveProject(o_orderkey=[$0], o_custkey=[$1], o_orderdate=[$4], o_shippriority=[$7]) HiveFilter(condition=[<($4, '1995-03-15')]) HiveTableScan(table=[[default.orders]]) 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Primary - Foreign Key join: fkSide = 1 FKInfo:FKInfo(rowCount=1.00,ndv=-1.00) PKInfo:PKInfo(rowCount=1.00,ndv=-1.00,selectivity=1.00) isPKSideSimple:false NDV Scaling Factor:1.00 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Identified Primary - Foreign Key relation: 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: HiveJoin(condition=[=($8, $5)], joinType=[inner]) HiveJoin(condition=[=($0, $4)], joinType=[inner]) HiveProject(l_orderkey=[$0], l_extendedprice=[$5], l_discount=[$6], l_shipdate=[$10]) HiveFilter(condition=[>($10, '1995-03-15')]) HiveTableScan(table=[[default.lineitem]]) HiveProject(o_orderkey=[$0], o_custkey=[$1], o_orderdate=[$4], o_shippriority=[$7]) HiveFilter(condition=[<($4, '1995-03-15')]) HiveTableScan(table=[[default.orders]]) HiveProject(c_custkey=[$0], c_mktsegment=[$6]) HiveFilter(condition=[=($6, 'BUILDING')]) HiveTableScan(table=[[default.customer]]) 15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Primary - Foreign Key join: fkSide = 1 FKInfo:FKInfo(rowCount=1.00,ndv=-1.00) PKInfo:PKInfo(rowCount=1.00,ndv=-1.00,selectivity=1.00) isPKSideSimple:false NDV Scaling Factor:1.00 Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.hep.HepPlanner dumpGraph FINER: Breadth-first from root: { HepRelVertex#197 = rel#196:HiveSort.HIVE.[](input=HepRelVertex#195,fetch=10), rowcount=1.0, cumulative cost={36.0 rows, 2.0 cpu, 0.0 io} HepRelVertex#195 = rel#194:HiveSort.HIVE.[1 DESC, 2](input=HepRelVertex#193,sort0=$1,sort1=$2,dir0=DESC,dir1=ASC), rowcount=1.0, cumulative cost={20.0 rows, 1.0 cpu, 0.0 io} HepRelVertex#193 = rel#192:HiveProject.HIVE.[](input=HepRelVertex#191,l_orderkey=$0,revenue=$3,o_orderdate=$1,o_shippriority=$2), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#191 = rel#190:HiveAggregate.HIVE.[](input=HepRelVertex#189,group={0, 1, 2},agg#0=sum($3)), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#189 = rel#188:HiveProject.HIVE.[](input=HepRelVertex#187,$f0=$0,$f1=$6,$f2=$7,$f3=*($1, -(CAST(1):DOUBLE, $2))), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#187 = rel#186:HiveJoin.HIVE.[](left=HepRelVertex#181,right=HepRelVertex#185,condition==($8, $5),joinType=inner), rowcount=1.0, cumulative cost={4.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#181 = rel#180:HiveJoin.HIVE.[](left=HepRelVertex#175,right=HepRelVertex#179,condition==($0, $4),joinType=inner), rowcount=1.0, cumulative cost={2.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#185 = rel#184:HiveProject.HIVE.[](input=HepRelVertex#183,c_custkey=$0,c_mktsegment=$6), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#175 = rel#174:HiveProject.HIVE.[](input=HepRelVertex#173,l_orderkey=$0,l_extendedprice=$5,l_discount=$6,l_shipdate=$10), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#179 = rel#178:HiveProject.HIVE.[](input=HepRelVertex#177,o_orderkey=$0,o_custkey=$1,o_orderdate=$4,o_shippriority=$7), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#183 = rel#182:HiveFilter.HIVE.[](input=HepRelVertex#155,condition==($6, 'BUILDING')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#173 = rel#172:HiveFilter.HIVE.[](input=HepRelVertex#143,condition=>($10, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#177 = rel#176:HiveFilter.HIVE.[](input=HepRelVertex#148,condition=<($4, '1995-03-15')), rowcount=1.0, cumulative cost={0.0 rows, 0.0 cpu, 0.0 io} HepRelVertex#155 = rel#2:HiveTableScan.HIVE.[](table=[default.customer]), rowcount=1.0, cumulative cost={0} HepRelVertex#143 = rel#0:HiveTableScan.HIVE.[](table=[default.lineitem]), rowcount=1.0, cumulative cost={0} HepRelVertex#148 = rel#1:HiveTableScan.HIVE.[](table=[default.orders]), rowcount=1.0, cumulative cost={0} } Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.hep.HepPlanner collectGarbage FINEST: collecting garbage Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule FINE: call#17: Apply rule [JoinToMultiJoinRule] to [rel#180:HiveJoin.HIVE.[](left=HepRelVertex#175,right=HepRelVertex#179,condition==($0, $4),joinType=inner), rel#174:HiveProject.HIVE.[](input=HepRelVertex#173,l_orderkey=$0,l_extendedprice=$5,l_discount=$6,l_shipdate=$10), rel#178:HiveProject.HIVE.[](input=HepRelVertex#177,o_orderkey=$0,o_custkey=$1,o_orderdate=$4,o_shippriority=$7)] Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new MultiJoin#201 Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner notifyTransformation FINE: call#17: Rule JoinToMultiJoinRule arguments [rel#180:HiveJoin.HIVE.[](left=HepRelVertex#175,right=HepRelVertex#179,condition==($0, $4),joinType=inner), rel#174:HiveProject.HIVE.[](input=HepRelVertex#173,l_orderkey=$0,l_extendedprice=$5,l_discount=$6,l_shipdate=$10), rel#178:HiveProject.HIVE.[](input=HepRelVertex#177,o_orderkey=$0,o_custkey=$1,o_orderdate=$4,o_shippriority=$7)] produced MultiJoin#201 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new MultiJoin#202 Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode FINEST: new HepRelVertex#203 15/01/29 11:48:04 [main]: ERROR parse.CalcitePlanner: CBO failed, skipping CBO. java.lang.NullPointerException at org.apache.calcite.rel.metadata.RelMdUtil.getJoinDistinctRowCount(RelMdUtil.java:653) at org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdDistinctRowCount.getDistinctRowCount(HiveRelMdDistinctRowCount.java:102) at sun.reflect.GeneratedMethodAccessor53.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1.invoke(ReflectiveRelMetadataProvider.java:147) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:314) at org.apache.calcite.rel.metadata.RelMdDistinctRowCount.getDistinctRowCount(RelMdDistinctRowCount.java:230) at sun.reflect.GeneratedMethodAccessor58.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1.invoke(ReflectiveRelMetadataProvider.java:147) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:314) at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:88) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1.invoke(ReflectiveRelMetadataProvider.java:147) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:81) at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:64) at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1.invoke(ReflectiveRelMetadataProvider.java:147) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:81) at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:68) at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1.invoke(ReflectiveRelMetadataProvider.java:147) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:81) at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:68) at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1.invoke(ReflectiveRelMetadataProvider.java:147) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:81) at org.apache.calcite.plan.hep.HepPlanner.dumpGraph(HepPlanner.java:958) at org.apache.calcite.plan.hep.HepPlanner.applyTransformationResults(HepPlanner.java:723) at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:518) at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:392) at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:255) at org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:125) at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:207) at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:194) at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:736) at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:658) at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:109) at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:691) at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:145) at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:105) at org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:568) at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:224) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9983) at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:190) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:420) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:306) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1106) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1154) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1043) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1033) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:207) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:159) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:370) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:305) at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:403) at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:419) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:708) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) FAILED: NullPointerException null 15/01/29 11:48:04 [main]: ERROR ql.Driver: FAILED: NullPointerException null java.lang.NullPointerException at org.apache.calcite.rel.metadata.RelMdUtil.getJoinDistinctRowCount(RelMdUtil.java:653) at org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdDistinctRowCount.getDistinctRowCount(HiveRelMdDistinctRowCount.java:102) at sun.reflect.GeneratedMethodAccessor53.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1.invoke(ReflectiveRelMetadataProvider.java:147) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:314) at org.apache.calcite.rel.metadata.RelMdDistinctRowCount.getDistinctRowCount(RelMdDistinctRowCount.java:230) at sun.reflect.GeneratedMethodAccessor58.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1.invoke(ReflectiveRelMetadataProvider.java:147) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy23.getDistinctRowCount(Unknown Source) at org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:314) at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:88) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1.invoke(ReflectiveRelMetadataProvider.java:147) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:81) at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:64) at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1.invoke(ReflectiveRelMetadataProvider.java:147) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:81) at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:68) at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1.invoke(ReflectiveRelMetadataProvider.java:147) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:81) at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:68) at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1.invoke(ReflectiveRelMetadataProvider.java:147) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:131) at com.sun.proxy.$Proxy22.getRowCount(Unknown Source) at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:81) at org.apache.calcite.plan.hep.HepPlanner.dumpGraph(HepPlanner.java:958) at org.apache.calcite.plan.hep.HepPlanner.applyTransformationResults(HepPlanner.java:723) at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:518) at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:392) at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:255) at org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:125) at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:207) at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:194) at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:736) at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:658) at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:109) at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:691) at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:145) at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:105) at org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:568) at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:224) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9983) at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:190) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:420) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:306) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1106) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1154) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1043) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1033) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:207) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:159) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:370) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:305) at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:403) at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:419) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:708) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) 15/01/29 11:48:04 [main]: INFO log.PerfLogger: 15/01/29 11:48:04 [main]: INFO log.PerfLogger: 15/01/29 11:48:04 [main]: INFO log.PerfLogger: 15/01/29 11:48:04 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun sending #11 15/01/29 11:48:04 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun got value #11 15/01/29 11:48:04 [main]: DEBUG ipc.ProtobufRpcEngine: Call: delete took 2ms 15/01/29 11:48:04 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun sending #12 15/01/29 11:48:04 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun got value #12 15/01/29 11:48:04 [main]: DEBUG ipc.ProtobufRpcEngine: Call: delete took 24ms 15/01/29 11:48:04 [main]: INFO log.PerfLogger: 15/01/29 11:48:04 [main]: INFO log.PerfLogger: 15/01/29 11:48:04 [main]: DEBUG session.SessionState: Removing resource dir /home/zhangyun/runtime/tmp/hive-0.15 15/01/29 11:48:04 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun sending #13 15/01/29 11:48:04 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun got value #13 15/01/29 11:48:04 [main]: DEBUG ipc.ProtobufRpcEngine: Call: delete took 8ms 15/01/29 11:48:04 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun sending #14 15/01/29 11:48:04 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun got value #14 15/01/29 11:48:04 [Thread-1]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 1ms 15/01/29 11:48:04 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun sending #15 15/01/29 11:48:04 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun got value #15 15/01/29 11:48:04 [Thread-1]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 1ms 15/01/29 11:48:04 [IPC Parameter Sending Thread #0]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun sending #16 15/01/29 11:48:04 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun got value #16 15/01/29 11:48:04 [Thread-1]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 1ms 15/01/29 11:48:04 [Thread-1]: DEBUG ipc.Client: Stopping client 15/01/29 11:48:04 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun: closed 15/01/29 11:48:04 [IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun]: DEBUG ipc.Client: IPC Client (716945906) connection to /10.3.1.76:29000 from zhangyun: stopped, remaining connections 0