[HIVE-24720] Got error while joining iceberg table and hive table - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: 2.0.1, 2.3.8
Fix Version/s: None
Component/s: Serializers/Deserializers, StorageHandler
Labels:
None
Environment:

Iceberg : 0.11

hive : 2.0.1

hadoop : 2.7.2

JDK : 1.8

Description

We got error while join iceberg table and hive table at same time, most of mappers succeed, but some mappers got an `cannot find field` error:

Caused by: java.lang.RuntimeException: cannot find field log_src from [org.apache.iceberg.mr.hive.serde.objectinspector.IcebergRecordObjectInspector$IcebergRecordStructField@8838736]
	at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:442)
	at org.apache.iceberg.mr.hive.serde.objectinspector.IcebergRecordObjectInspector.getStructFieldRef(IcebergRecordObjectInspector.java:78)
	at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:55)
	at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:139)
	at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:980)
	at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:1006)
	at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:77)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:355)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:504)
	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:457)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:365)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:504)
	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:457)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:365)
	at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:498)
	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:115)
	... 22 more

2021-02-02 15:57:35,036 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: Paths:/tmp/hive/flink/25369d0b-110e-4eca-8553-f2f3953f0303/hive_2021-02-02_15-56-22_060_2459779186626852210-1/-mr-10005/0/emptyFile:0+0InputFormatClass: org.apache.hadoop.mapred.TextInputFormat

The split file is from `hive table`(which is an empty table), but still the mapper using `org.apache.iceberg.mr.hive.HiveIcebergSerDe`, the table properties is also from iceberg table.

It seems like hive use wrong serd to deserialize data, I am not an expert in hive, hope someone could give me some clues .

SQL:

select count(1),count(a.dt),count(b.dt)
from (
select dt,concat_ws('###',wx_source, log_src, dt, hour) as str
from flink_fdm_iceberg.iceberg_table1
where dt='2021-01-29' and hour='16') a
full outer join (
select dt,concat_ws('###',wx_source, log_src, dt, hour) as str
from flink_fdm_iceberg.hive_table1
where dt='2021-01-29' and hour='16') b on a.str=b.str
where a.str is null or b.str is null;

The full log by the failed mapper:

2021-02-02 15:57:30,060 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties2021-02-02 15:57:30,060 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties2021-02-02 15:57:30,118 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).2021-02-02 15:57:30,118 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started2021-02-02 15:57:30,131 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens:2021-02-02 15:57:30,131 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1605975810748_1427, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@4b2bac3f)2021-02-02 15:57:30,167 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:ns2, Ident: (HDFS_DELEGATION_TOKEN token 151282235 for flink)2021-02-02 15:57:30,168 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:ns1, Ident: (HDFS_DELEGATION_TOKEN token 151249944 for flink)2021-02-02 15:57:30,168 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:ns4, Ident: (HDFS_DELEGATION_TOKEN token 151235141 for flink)2021-02-02 15:57:30,169 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:ns3, Ident: (HDFS_DELEGATION_TOKEN token 151183885 for flink)2021-02-02 15:57:30,169 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:ns6, Ident: (HDFS_DELEGATION_TOKEN token 168039224 for flink)2021-02-02 15:57:30,169 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:ns5, Ident: (HDFS_DELEGATION_TOKEN token 150801000 for flink)2021-02-02 15:57:30,881 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.2021-02-02 15:57:31,032 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /data1/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data2/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data4/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data5/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data6/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data7/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data8/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data9/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data10/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data11/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data12/nm-local-dir/usercache/flink/appcache/application_1605975810748_14272021-02-02 15:57:31,937 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id2021-02-02 15:57:34,143 INFO [main] org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]2021-02-02 15:57:35,036 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: Paths:/tmp/hive/flink/25369d0b-110e-4eca-8553-f2f3953f0303/hive_2021-02-02_15-56-22_060_2459779186626852210-1/-mr-10005/0/emptyFile:0+0InputFormatClass: org.apache.hadoop.mapred.TextInputFormat
2021-02-02 15:57:35,198 INFO [main] org.apache.hadoop.hive.ql.exec.Utilities: PLAN PATH = hdfs://ns6/tmp/hive/flink/25369d0b-110e-4eca-8553-f2f3953f0303/hive_2021-02-02_15-56-22_060_2459779186626852210-1/-mr-10006/33cfad1a-7384-4c27-b2d4-2f894ecb845a/map.xml2021-02-02 15:57:35,209 WARN [main] org.apache.hadoop.ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category READ is not supported in state standby2021-02-02 15:57:35,221 INFO [main] org.apache.hadoop.hive.ql.exec.SerializationUtilities: Deserializing MapWork using kryo2021-02-02 15:57:35,384 INFO [main] com.hadoop.compression.lzo.GPLNativeCodeLoader: Loaded native gpl library from the embedded binaries2021-02-02 15:57:35,386 INFO [main] com.hadoop.compression.lzo.LzoCodec: Successfully loaded & initialized native-lzo library [hadoop-lzo rev 5e360bdefd8923280b1a0234b845448050bf0caa]2021-02-02 15:57:35,857 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: map.input.file is deprecated. Instead, use mapreduce.map.input.file2021-02-02 15:57:35,857 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: map.input.start is deprecated. Instead, use mapreduce.map.input.start2021-02-02 15:57:35,857 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: map.input.length is deprecated. Instead, use mapreduce.map.input.length2021-02-02 15:57:35,857 INFO [main] org.apache.hadoop.mapred.MapTask: numReduceTasks: 10092021-02-02 15:57:37,109 INFO [main] org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 268435452(1073741808)2021-02-02 15:57:37,110 INFO [main] org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 10242021-02-02 15:57:37,110 INFO [main] org.apache.hadoop.mapred.MapTask: soft limit at 8589934722021-02-02 15:57:37,110 INFO [main] org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 10737418242021-02-02 15:57:37,110 INFO [main] org.apache.hadoop.mapred.MapTask: kvstart = 268435452; length = 671088642021-02-02 15:57:37,114 INFO [main] org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer2021-02-02 15:57:37,119 INFO [main] org.apache.hadoop.hive.ql.exec.mr.ExecMapper: conf classpath = [file:/data12/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/container_e03_1605975810748_1427_01_000559/, file:/data/sysdir/servers/hadoop-2.7.2-yarn/etc/hadoop/, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/hadoop-nfs-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/hadoop-lzo-0.4.20-SNAPSHOT.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/mysql-connector-java-5.1.31-bin.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/spark-2.0.2-yarn-shuffle.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/hadoop-common-2.7.2-tests.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/data-hadoop-patch.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/hadoop-common-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jaxb-api-2.2.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/curator-client-2.7.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/netty-3.6.2.Final.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jetty-6.1.26.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jetty-util-6.1.26.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/junit-4.11.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/avro-1.7.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jets3t-0.9.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-httpclient-3.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-math3-3.1.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/servlet-api-2.5.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-logging-1.1.3.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/httpclient-4.2.5.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/protobuf-java-2.5.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/mockito-all-1.8.5.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/hamcrest-core-1.3.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/curator-recipes-2.7.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-collections-3.2.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/htrace-core-3.1.0-incubating.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jersey-server-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/guava-11.0.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/hadoop-annotations-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/httpcore-4.2.5.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-io-2.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jsp-api-2.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jersey-json-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/zookeeper-3.4.6.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/hadoop-auth-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-cli-1.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jersey-core-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-lang-2.6.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/slf4j-api-1.7.10.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/xmlenc-0.52.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jettison-1.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/api-util-1.0.0-M20.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/asm-3.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/snappy-java-1.0.4.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jsch-0.1.42.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/stax-api-1.0-2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jackson-xc-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/paranamer-2.3.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-configuration-1.6.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/log4j-1.2.17.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-compress-1.4.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/gson-2.2.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jsr305-3.0.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-beanutils-1.7.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/java-xmlbuilder-0.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-net-3.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-digester-1.8.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/curator-framework-2.7.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/xz-1.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/activation-1.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-codec-1.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/hadoop-hdfs-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/hadoop-hdfs-nfs-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/hadoop-hdfs-2.7.2-tests.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/ranger-plugin-classloader-1.0.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jetty-6.1.26.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/ranger-hdfs-plugin-shim-1.0.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/servlet-api-2.5.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/htrace-core-3.1.0-incubating.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jersey-server-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/guava-11.0.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-io-2.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-cli-1.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jersey-core-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-lang-2.6.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/xmlenc-0.52.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/asm-3.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/log4j-1.2.17.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jsr305-3.0.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-codec-1.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-client-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-common-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-common-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-registry-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-tests-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-api-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/netty-3.6.2.Final.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jetty-6.1.26.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jetty-util-6.1.26.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/guice-3.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/servlet-api-2.5.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-logging-1.1.3.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/guice-servlet-3.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-collections-3.2.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-server-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/guava-11.0.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/aopalliance-1.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-io-2.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-json-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-client-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/zookeeper-3.4.6.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-cli-1.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-core-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-lang-2.6.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jettison-1.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/asm-3.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/stax-api-1.0-2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/log4j-1.2.17.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/javax.inject-1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-compress-1.4.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jsr305-3.0.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-guice-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/zookeeper-3.4.6-tests.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/xz-1.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/activation-1.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-codec-1.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2-tests.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/junit-4.11.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/avro-1.7.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/guice-3.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jersey-server-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/hadoop-annotations-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/aopalliance-1.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/commons-io-2.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jersey-core-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/asm-3.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/paranamer-2.3.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/log4j-1.2.17.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/javax.inject-1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/xz-1.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar, file:/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/filecache/10/job.jar/job.jar, file:/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/filecache/10/job.jar/classes, file:/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/filecache/10/job.jar/lib/*, file:/data2/nm-local-dir/usercache/flink/filecache/299/UserAgentUtils-1.19.jar, file:/data9/nm-local-dir/usercache/flink/filecache/294/iceberg-hive-runtime-022dc46.dirty.jar, file:/data7/nm-local-dir/usercache/flink/filecache/292/hive-jdbc-1.2.1.jar, file:/data12/nm-local-dir/usercache/flink/filecache/297/dc-ua-util-2.0.0-SNAPSHOT.jar, file:/data11/nm-local-dir/usercache/flink/filecache/296/check-hive-db.jar, file:/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/filecache/10/job.jar/, file:/data10/nm-local-dir/usercache/flink/filecache/295/data-udf-0.0.1.jar, file:/data6/nm-local-dir/usercache/flink/filecache/291/sqljdbc4-4.0.jar]2021-02-02 15:57:37,121 INFO [main] org.apache.hadoop.hive.ql.exec.mr.ExecMapper: thread classpath = [file:/data12/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/container_e03_1605975810748_1427_01_000559/, file:/data/sysdir/servers/hadoop-2.7.2-yarn/etc/hadoop/, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/hadoop-nfs-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/hadoop-lzo-0.4.20-SNAPSHOT.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/mysql-connector-java-5.1.31-bin.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/spark-2.0.2-yarn-shuffle.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/hadoop-common-2.7.2-tests.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/data-hadoop-patch.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/hadoop-common-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jaxb-api-2.2.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/curator-client-2.7.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/netty-3.6.2.Final.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jetty-6.1.26.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jetty-util-6.1.26.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/junit-4.11.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/avro-1.7.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jets3t-0.9.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-httpclient-3.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-math3-3.1.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/servlet-api-2.5.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-logging-1.1.3.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/httpclient-4.2.5.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/protobuf-java-2.5.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/mockito-all-1.8.5.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/hamcrest-core-1.3.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/curator-recipes-2.7.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-collections-3.2.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/htrace-core-3.1.0-incubating.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jersey-server-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/guava-11.0.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/hadoop-annotations-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/httpcore-4.2.5.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-io-2.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jsp-api-2.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jersey-json-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/zookeeper-3.4.6.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/hadoop-auth-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-cli-1.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jersey-core-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-lang-2.6.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/slf4j-api-1.7.10.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/xmlenc-0.52.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jettison-1.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/api-util-1.0.0-M20.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/asm-3.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/snappy-java-1.0.4.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jsch-0.1.42.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/stax-api-1.0-2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jackson-xc-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/paranamer-2.3.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-configuration-1.6.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/log4j-1.2.17.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-compress-1.4.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/gson-2.2.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jsr305-3.0.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-beanutils-1.7.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/java-xmlbuilder-0.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-net-3.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-digester-1.8.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/curator-framework-2.7.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/xz-1.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/activation-1.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-codec-1.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/hadoop-hdfs-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/hadoop-hdfs-nfs-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/hadoop-hdfs-2.7.2-tests.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/ranger-plugin-classloader-1.0.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jetty-6.1.26.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/ranger-hdfs-plugin-shim-1.0.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/servlet-api-2.5.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/htrace-core-3.1.0-incubating.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jersey-server-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/guava-11.0.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-io-2.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-cli-1.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jersey-core-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-lang-2.6.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/xmlenc-0.52.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/asm-3.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/log4j-1.2.17.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jsr305-3.0.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-codec-1.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-client-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-common-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-common-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-registry-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-tests-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-api-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/netty-3.6.2.Final.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jetty-6.1.26.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jetty-util-6.1.26.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/guice-3.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/servlet-api-2.5.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-logging-1.1.3.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/guice-servlet-3.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-collections-3.2.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-server-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/guava-11.0.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/aopalliance-1.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-io-2.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-json-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-client-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/zookeeper-3.4.6.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-cli-1.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-core-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-lang-2.6.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jettison-1.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/asm-3.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/stax-api-1.0-2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/log4j-1.2.17.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/javax.inject-1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-compress-1.4.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jsr305-3.0.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-guice-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/zookeeper-3.4.6-tests.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/xz-1.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/activation-1.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-codec-1.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2-tests.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/junit-4.11.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/avro-1.7.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/guice-3.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jersey-server-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/hadoop-annotations-2.7.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/aopalliance-1.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/commons-io-2.4.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jersey-core-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/asm-3.2.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/paranamer-2.3.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/log4j-1.2.17.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/javax.inject-1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/xz-1.0.jar, file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar, file:/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/filecache/10/job.jar/job.jar, file:/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/filecache/10/job.jar/classes, file:/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/filecache/10/job.jar/lib/*, file:/data2/nm-local-dir/usercache/flink/filecache/299/UserAgentUtils-1.19.jar, file:/data9/nm-local-dir/usercache/flink/filecache/294/iceberg-hive-runtime-022dc46.dirty.jar, file:/data7/nm-local-dir/usercache/flink/filecache/292/hive-jdbc-1.2.1.jar, file:/data12/nm-local-dir/usercache/flink/filecache/297/dc-ua-util-2.0.0-SNAPSHOT.jar, file:/data11/nm-local-dir/usercache/flink/filecache/296/check-hive-db.jar, file:/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/filecache/10/job.jar/, file:/data10/nm-local-dir/usercache/flink/filecache/295/data-udf-0.0.1.jar, file:/data6/nm-local-dir/usercache/flink/filecache/291/sqljdbc4-4.0.jar]2021-02-02 15:57:37,121 INFO [main] org.apache.hadoop.hive.ql.exec.Utilities: PLAN PATH = hdfs://ns6/tmp/hive/flink/25369d0b-110e-4eca-8553-f2f3953f0303/hive_2021-02-02_15-56-22_060_2459779186626852210-1/-mr-10006/33cfad1a-7384-4c27-b2d4-2f894ecb845a/map.xml2021-02-02 15:57:37,123 INFO [main] org.apache.hadoop.hive.ql.exec.MapOperator: Initializing operator MAP[0]2021-02-02 15:57:37,128 INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe: serDeProperties : {columns.types=string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string, location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1, columns=version,load_id,pre_load_id,sign,visit_time,ip,province_id,city_id,county_id,appkey,session_id,user_id,ua,visits_id,first_visit_time,first_visit_load_id,first_visit_ref_domain,first_visit_subdomain_time,first_visit_subdomain_load_id,first_visit_subdomain_refer,visits_ref,visits_keyword,visits_area_id,bt_visits_ref,last_visit_subdomain_refer,site_ref,scene_type,show_id,ref,refer_domain,key_word,area_id,current_url,current_domain,site_id,category_id,sub_category_id,object_id,series_id,spec_id,dealer_id,level_id,doc_title,page_custom_vars,platform,screen,color_depth,browser_charset,browser_language,browser_cookie_enable,extend,create_time,id,custom1,custom2,customer3,customer4,longitude,latitude,browse_type,ac_userid,ac_app_key,ac_app_platform,ac_app_ver,ac_app_deviceid,cur_visit_ref_domain,cookie_content,pre_visits_ref,lt_visits_ref,abtest,wx_openid,wx_source,log_src,dt,hour, metadata_location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1/metadata/00753-b2322cf0-ac32-4eb0-957e-95d2189e0272.metadata.json, previous_metadata_location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1/metadata/00752-6949a171-403e-4853-93a3-666df4305603.metadata.json, numFiles=59177, serialization.ddl=struct icebert_table1 { string version, string load_id, string pre_load_id, string sign, string visit_time, string ip, string province_id, string city_id, string county_id, string appkey, string session_id, string user_id, string ua, string visits_id, string first_visit_time, string first_visit_load_id, string first_visit_ref_domain, string first_visit_subdomain_time, string first_visit_subdomain_load_id, string first_visit_subdomain_refer, string visits_ref, string visits_keyword, string visits_area_id, string bt_visits_ref, string last_visit_subdomain_refer, string site_ref, string scene_type, string show_id, string ref, string refer_domain, string key_word, string area_id, string current_url, string current_domain, string site_id, string category_id, string sub_category_id, string object_id, string series_id, string spec_id, string dealer_id, string level_id, string doc_title, string page_custom_vars, string platform, string screen, string color_depth, string browser_charset, string browser_language, string browser_cookie_enable, string extend, string create_time, string id, string custom1, string custom2, string customer3, string customer4, string longitude, string latitude, string browse_type, string ac_userid, string ac_app_key, string ac_app_platform, string ac_app_ver, string ac_app_deviceid, string cur_visit_ref_domain, string cookie_content, string pre_visits_ref, string lt_visits_ref, string abtest, string wx_openid, string wx_source, string log_src, string dt, string hour}, transient_lastDdlTime=1611904166, iceberg.mr.table.schema={"type":"struct","fields":[{"id":1,"name":"version","required":false,"type":"string"},{"id":2,"name":"load_id","required":false,"type":"string"},{"id":3,"name":"pre_load_id","required":false,"type":"string"},{"id":4,"name":"sign","required":false,"type":"string"},{"id":5,"name":"visit_time","required":false,"type":"string"},{"id":6,"name":"ip","required":false,"type":"string"},{"id":7,"name":"province_id","required":false,"type":"string"},{"id":8,"name":"city_id","required":false,"type":"string"},{"id":9,"name":"county_id","required":false,"type":"string"},{"id":10,"name":"appkey","required":false,"type":"string"},{"id":11,"name":"session_id","required":false,"type":"string"},{"id":12,"name":"user_id","required":false,"type":"string"},{"id":13,"name":"ua","required":false,"type":"string"},{"id":14,"name":"visits_id","required":false,"type":"string"},{"id":15,"name":"first_visit_time","required":false,"type":"string"},{"id":16,"name":"first_visit_load_id","required":false,"type":"string"},{"id":17,"name":"first_visit_ref_domain","required":false,"type":"string"},{"id":18,"name":"first_visit_subdomain_time","required":false,"type":"string"},{"id":19,"name":"first_visit_subdomain_load_id","required":false,"type":"string"},{"id":20,"name":"first_visit_subdomain_refer","required":false,"type":"string"},{"id":21,"name":"visits_ref","required":false,"type":"string"},{"id":22,"name":"visits_keyword","required":false,"type":"string"},{"id":23,"name":"visits_area_id","required":false,"type":"string"},{"id":24,"name":"bt_visits_ref","required":false,"type":"string"},{"id":25,"name":"last_visit_subdomain_refer","required":false,"type":"string"},{"id":26,"name":"site_ref","required":false,"type":"string"},{"id":27,"name":"scene_type","required":false,"type":"string"},{"id":28,"name":"show_id","required":false,"type":"string"},{"id":29,"name":"ref","required":false,"type":"string"},{"id":30,"name":"refer_domain","required":false,"type":"string"},{"id":31,"name":"key_word","required":false,"type":"string"},{"id":32,"name":"area_id","required":false,"type":"string"},{"id":33,"name":"current_url","required":false,"type":"string"},{"id":34,"name":"current_domain","required":false,"type":"string"},{"id":35,"name":"site_id","required":false,"type":"string"},{"id":36,"name":"category_id","required":false,"type":"string"},{"id":37,"name":"sub_category_id","required":false,"type":"string"},{"id":38,"name":"object_id","required":false,"type":"string"},{"id":39,"name":"series_id","required":false,"type":"string"},{"id":40,"name":"spec_id","required":false,"type":"string"},{"id":41,"name":"dealer_id","required":false,"type":"string"},{"id":42,"name":"level_id","required":false,"type":"string"},{"id":43,"name":"doc_title","required":false,"type":"string"},{"id":44,"name":"page_custom_vars","required":false,"type":"string"},{"id":45,"name":"platform","required":false,"type":"string"},{"id":46,"name":"screen","required":false,"type":"string"},{"id":47,"name":"color_depth","required":false,"type":"string"},{"id":48,"name":"browser_charset","required":false,"type":"string"},{"id":49,"name":"browser_language","required":false,"type":"string"},{"id":50,"name":"browser_cookie_enable","required":false,"type":"string"},{"id":51,"name":"extend","required":false,"type":"string"},{"id":52,"name":"create_time","required":false,"type":"string"},{"id":53,"name":"id","required":false,"type":"string"},{"id":54,"name":"custom1","required":false,"type":"string"},{"id":55,"name":"custom2","required":false,"type":"string"},{"id":56,"name":"customer3","required":false,"type":"string"},{"id":57,"name":"customer4","required":false,"type":"string"},{"id":58,"name":"longitude","required":false,"type":"string"},{"id":59,"name":"latitude","required":false,"type":"string"},{"id":60,"name":"browse_type","required":false,"type":"string"},{"id":61,"name":"ac_userid","required":false,"type":"string"},{"id":62,"name":"ac_app_key","required":false,"type":"string"},{"id":63,"name":"ac_app_platform","required":false,"type":"string"},{"id":64,"name":"ac_app_ver","required":false,"type":"string"},{"id":65,"name":"ac_app_deviceid","required":false,"type":"string"},{"id":66,"name":"cur_visit_ref_domain","required":false,"type":"string"},{"id":67,"name":"cookie_content","required":false,"type":"string"},{"id":68,"name":"pre_visits_ref","required":false,"type":"string"},{"id":69,"name":"lt_visits_ref","required":false,"type":"string"},{"id":70,"name":"abtest","required":false,"type":"string"},{"id":71,"name":"wx_openid","required":false,"type":"string"},{"id":72,"name":"wx_source","required":false,"type":"string"},{"id":73,"name":"log_src","required":false,"type":"string"},{"id":74,"name":"dt","required":false,"type":"string"},{"id":75,"name":"hour","required":false,"type":"string"}]}, columns.comments=, totalSize=302458258710, bucket_count=0, file.outputformat=org.apache.iceberg.mr.hive.HiveIcebergOutputFormat, serialization.lib=org.apache.iceberg.mr.hive.HiveIcebergSerDe, storage_handler=org.apache.iceberg.mr.hive.HiveIcebergStorageHandler, table_type=ICEBERG, file.inputformat=org.apache.iceberg.mr.hive.HiveIcebergInputFormat, EXTERNAL=TRUE, name=flink_fdm_iceberg.icebert_table1}2021-02-02 15:57:37,906 INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe: Using schema from serDeProperties {"type":"struct","fields":[{"id":1,"name":"version","required":false,"type":"string"},{"id":2,"name":"load_id","required":false,"type":"string"},{"id":3,"name":"pre_load_id","required":false,"type":"string"},{"id":4,"name":"sign","required":false,"type":"string"},{"id":5,"name":"visit_time","required":false,"type":"string"},{"id":6,"name":"ip","required":false,"type":"string"},{"id":7,"name":"province_id","required":false,"type":"string"},{"id":8,"name":"city_id","required":false,"type":"string"},{"id":9,"name":"county_id","required":false,"type":"string"},{"id":10,"name":"appkey","required":false,"type":"string"},{"id":11,"name":"session_id","required":false,"type":"string"},{"id":12,"name":"user_id","required":false,"type":"string"},{"id":13,"name":"ua","required":false,"type":"string"},{"id":14,"name":"visits_id","required":false,"type":"string"},{"id":15,"name":"first_visit_time","required":false,"type":"string"},{"id":16,"name":"first_visit_load_id","required":false,"type":"string"},{"id":17,"name":"first_visit_ref_domain","required":false,"type":"string"},{"id":18,"name":"first_visit_subdomain_time","required":false,"type":"string"},{"id":19,"name":"first_visit_subdomain_load_id","required":false,"type":"string"},{"id":20,"name":"first_visit_subdomain_refer","required":false,"type":"string"},{"id":21,"name":"visits_ref","required":false,"type":"string"},{"id":22,"name":"visits_keyword","required":false,"type":"string"},{"id":23,"name":"visits_area_id","required":false,"type":"string"},{"id":24,"name":"bt_visits_ref","required":false,"type":"string"},{"id":25,"name":"last_visit_subdomain_refer","required":false,"type":"string"},{"id":26,"name":"site_ref","required":false,"type":"string"},{"id":27,"name":"scene_type","required":false,"type":"string"},{"id":28,"name":"show_id","required":false,"type":"string"},{"id":29,"name":"ref","required":false,"type":"string"},{"id":30,"name":"refer_domain","required":false,"type":"string"},{"id":31,"name":"key_word","required":false,"type":"string"},{"id":32,"name":"area_id","required":false,"type":"string"},{"id":33,"name":"current_url","required":false,"type":"string"},{"id":34,"name":"current_domain","required":false,"type":"string"},{"id":35,"name":"site_id","required":false,"type":"string"},{"id":36,"name":"category_id","required":false,"type":"string"},{"id":37,"name":"sub_category_id","required":false,"type":"string"},{"id":38,"name":"object_id","required":false,"type":"string"},{"id":39,"name":"series_id","required":false,"type":"string"},{"id":40,"name":"spec_id","required":false,"type":"string"},{"id":41,"name":"dealer_id","required":false,"type":"string"},{"id":42,"name":"level_id","required":false,"type":"string"},{"id":43,"name":"doc_title","required":false,"type":"string"},{"id":44,"name":"page_custom_vars","required":false,"type":"string"},{"id":45,"name":"platform","required":false,"type":"string"},{"id":46,"name":"screen","required":false,"type":"string"},{"id":47,"name":"color_depth","required":false,"type":"string"},{"id":48,"name":"browser_charset","required":false,"type":"string"},{"id":49,"name":"browser_language","required":false,"type":"string"},{"id":50,"name":"browser_cookie_enable","required":false,"type":"string"},{"id":51,"name":"extend","required":false,"type":"string"},{"id":52,"name":"create_time","required":false,"type":"string"},{"id":53,"name":"id","required":false,"type":"string"},{"id":54,"name":"custom1","required":false,"type":"string"},{"id":55,"name":"custom2","required":false,"type":"string"},{"id":56,"name":"customer3","required":false,"type":"string"},{"id":57,"name":"customer4","required":false,"type":"string"},{"id":58,"name":"longitude","required":false,"type":"string"},{"id":59,"name":"latitude","required":false,"type":"string"},{"id":60,"name":"browse_type","required":false,"type":"string"},{"id":61,"name":"ac_userid","required":false,"type":"string"},{"id":62,"name":"ac_app_key","required":false,"type":"string"},{"id":63,"name":"ac_app_platform","required":false,"type":"string"},{"id":64,"name":"ac_app_ver","required":false,"type":"string"},{"id":65,"name":"ac_app_deviceid","required":false,"type":"string"},{"id":66,"name":"cur_visit_ref_domain","required":false,"type":"string"},{"id":67,"name":"cookie_content","required":false,"type":"string"},{"id":68,"name":"pre_visits_ref","required":false,"type":"string"},{"id":69,"name":"lt_visits_ref","required":false,"type":"string"},{"id":70,"name":"abtest","required":false,"type":"string"},{"id":71,"name":"wx_openid","required":false,"type":"string"},{"id":72,"name":"wx_source","required":false,"type":"string"},{"id":73,"name":"log_src","required":false,"type":"string"},{"id":74,"name":"dt","required":false,"type":"string"},{"id":75,"name":"hour","required":false,"type":"string"}]}2021-02-02 15:57:37,907 INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe: selectedColumns 1, [wx_source]2021-02-02 15:57:37,926 INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe: Using projectedSchema {"type":"struct","fields":[{"id":72,"name":"wx_source","required":false,"type":"string"}]}2021-02-02 15:57:37,969 INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe: serDeProperties : {columns.types=string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string, location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1, columns=version,load_id,pre_load_id,sign,visit_time,ip,province_id,city_id,county_id,appkey,session_id,user_id,ua,visits_id,first_visit_time,first_visit_load_id,first_visit_ref_domain,first_visit_subdomain_time,first_visit_subdomain_load_id,first_visit_subdomain_refer,visits_ref,visits_keyword,visits_area_id,bt_visits_ref,last_visit_subdomain_refer,site_ref,scene_type,show_id,ref,refer_domain,key_word,area_id,current_url,current_domain,site_id,category_id,sub_category_id,object_id,series_id,spec_id,dealer_id,level_id,doc_title,page_custom_vars,platform,screen,color_depth,browser_charset,browser_language,browser_cookie_enable,extend,create_time,id,custom1,custom2,customer3,customer4,longitude,latitude,browse_type,ac_userid,ac_app_key,ac_app_platform,ac_app_ver,ac_app_deviceid,cur_visit_ref_domain,cookie_content,pre_visits_ref,lt_visits_ref,abtest,wx_openid,wx_source,log_src,dt,hour, metadata_location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1/metadata/00753-b2322cf0-ac32-4eb0-957e-95d2189e0272.metadata.json, previous_metadata_location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1/metadata/00752-6949a171-403e-4853-93a3-666df4305603.metadata.json, numFiles=59177, serialization.ddl=struct icebert_table1 { string version, string load_id, string pre_load_id, string sign, string visit_time, string ip, string province_id, string city_id, string county_id, string appkey, string session_id, string user_id, string ua, string visits_id, string first_visit_time, string first_visit_load_id, string first_visit_ref_domain, string first_visit_subdomain_time, string first_visit_subdomain_load_id, string first_visit_subdomain_refer, string visits_ref, string visits_keyword, string visits_area_id, string bt_visits_ref, string last_visit_subdomain_refer, string site_ref, string scene_type, string show_id, string ref, string refer_domain, string key_word, string area_id, string current_url, string current_domain, string site_id, string category_id, string sub_category_id, string object_id, string series_id, string spec_id, string dealer_id, string level_id, string doc_title, string page_custom_vars, string platform, string screen, string color_depth, string browser_charset, string browser_language, string browser_cookie_enable, string extend, string create_time, string id, string custom1, string custom2, string customer3, string customer4, string longitude, string latitude, string browse_type, string ac_userid, string ac_app_key, string ac_app_platform, string ac_app_ver, string ac_app_deviceid, string cur_visit_ref_domain, string cookie_content, string pre_visits_ref, string lt_visits_ref, string abtest, string wx_openid, string wx_source, string log_src, string dt, string hour}, transient_lastDdlTime=1611904166, iceberg.mr.table.schema={"type":"struct","fields":[{"id":1,"name":"version","required":false,"type":"string"},{"id":2,"name":"load_id","required":false,"type":"string"},{"id":3,"name":"pre_load_id","required":false,"type":"string"},{"id":4,"name":"sign","required":false,"type":"string"},{"id":5,"name":"visit_time","required":false,"type":"string"},{"id":6,"name":"ip","required":false,"type":"string"},{"id":7,"name":"province_id","required":false,"type":"string"},{"id":8,"name":"city_id","required":false,"type":"string"},{"id":9,"name":"county_id","required":false,"type":"string"},{"id":10,"name":"appkey","required":false,"type":"string"},{"id":11,"name":"session_id","required":false,"type":"string"},{"id":12,"name":"user_id","required":false,"type":"string"},{"id":13,"name":"ua","required":false,"type":"string"},{"id":14,"name":"visits_id","required":false,"type":"string"},{"id":15,"name":"first_visit_time","required":false,"type":"string"},{"id":16,"name":"first_visit_load_id","required":false,"type":"string"},{"id":17,"name":"first_visit_ref_domain","required":false,"type":"string"},{"id":18,"name":"first_visit_subdomain_time","required":false,"type":"string"},{"id":19,"name":"first_visit_subdomain_load_id","required":false,"type":"string"},{"id":20,"name":"first_visit_subdomain_refer","required":false,"type":"string"},{"id":21,"name":"visits_ref","required":false,"type":"string"},{"id":22,"name":"visits_keyword","required":false,"type":"string"},{"id":23,"name":"visits_area_id","required":false,"type":"string"},{"id":24,"name":"bt_visits_ref","required":false,"type":"string"},{"id":25,"name":"last_visit_subdomain_refer","required":false,"type":"string"},{"id":26,"name":"site_ref","required":false,"type":"string"},{"id":27,"name":"scene_type","required":false,"type":"string"},{"id":28,"name":"show_id","required":false,"type":"string"},{"id":29,"name":"ref","required":false,"type":"string"},{"id":30,"name":"refer_domain","required":false,"type":"string"},{"id":31,"name":"key_word","required":false,"type":"string"},{"id":32,"name":"area_id","required":false,"type":"string"},{"id":33,"name":"current_url","required":false,"type":"string"},{"id":34,"name":"current_domain","required":false,"type":"string"},{"id":35,"name":"site_id","required":false,"type":"string"},{"id":36,"name":"category_id","required":false,"type":"string"},{"id":37,"name":"sub_category_id","required":false,"type":"string"},{"id":38,"name":"object_id","required":false,"type":"string"},{"id":39,"name":"series_id","required":false,"type":"string"},{"id":40,"name":"spec_id","required":false,"type":"string"},{"id":41,"name":"dealer_id","required":false,"type":"string"},{"id":42,"name":"level_id","required":false,"type":"string"},{"id":43,"name":"doc_title","required":false,"type":"string"},{"id":44,"name":"page_custom_vars","required":false,"type":"string"},{"id":45,"name":"platform","required":false,"type":"string"},{"id":46,"name":"screen","required":false,"type":"string"},{"id":47,"name":"color_depth","required":false,"type":"string"},{"id":48,"name":"browser_charset","required":false,"type":"string"},{"id":49,"name":"browser_language","required":false,"type":"string"},{"id":50,"name":"browser_cookie_enable","required":false,"type":"string"},{"id":51,"name":"extend","required":false,"type":"string"},{"id":52,"name":"create_time","required":false,"type":"string"},{"id":53,"name":"id","required":false,"type":"string"},{"id":54,"name":"custom1","required":false,"type":"string"},{"id":55,"name":"custom2","required":false,"type":"string"},{"id":56,"name":"customer3","required":false,"type":"string"},{"id":57,"name":"customer4","required":false,"type":"string"},{"id":58,"name":"longitude","required":false,"type":"string"},{"id":59,"name":"latitude","required":false,"type":"string"},{"id":60,"name":"browse_type","required":false,"type":"string"},{"id":61,"name":"ac_userid","required":false,"type":"string"},{"id":62,"name":"ac_app_key","required":false,"type":"string"},{"id":63,"name":"ac_app_platform","required":false,"type":"string"},{"id":64,"name":"ac_app_ver","required":false,"type":"string"},{"id":65,"name":"ac_app_deviceid","required":false,"type":"string"},{"id":66,"name":"cur_visit_ref_domain","required":false,"type":"string"},{"id":67,"name":"cookie_content","required":false,"type":"string"},{"id":68,"name":"pre_visits_ref","required":false,"type":"string"},{"id":69,"name":"lt_visits_ref","required":false,"type":"string"},{"id":70,"name":"abtest","required":false,"type":"string"},{"id":71,"name":"wx_openid","required":false,"type":"string"},{"id":72,"name":"wx_source","required":false,"type":"string"},{"id":73,"name":"log_src","required":false,"type":"string"},{"id":74,"name":"dt","required":false,"type":"string"},{"id":75,"name":"hour","required":false,"type":"string"}]}, columns.comments=, totalSize=302458258710, bucket_count=0, file.outputformat=org.apache.iceberg.mr.hive.HiveIcebergOutputFormat, serialization.lib=org.apache.iceberg.mr.hive.HiveIcebergSerDe, storage_handler=org.apache.iceberg.mr.hive.HiveIcebergStorageHandler, table_type=ICEBERG, file.inputformat=org.apache.iceberg.mr.hive.HiveIcebergInputFormat, EXTERNAL=TRUE, name=flink_fdm_iceberg.icebert_table1}2021-02-02 15:57:37,973 INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe: Using schema from serDeProperties {"type":"struct","fields":[{"id":1,"name":"version","required":false,"type":"string"},{"id":2,"name":"load_id","required":false,"type":"string"},{"id":3,"name":"pre_load_id","required":false,"type":"string"},{"id":4,"name":"sign","required":false,"type":"string"},{"id":5,"name":"visit_time","required":false,"type":"string"},{"id":6,"name":"ip","required":false,"type":"string"},{"id":7,"name":"province_id","required":false,"type":"string"},{"id":8,"name":"city_id","required":false,"type":"string"},{"id":9,"name":"county_id","required":false,"type":"string"},{"id":10,"name":"appkey","required":false,"type":"string"},{"id":11,"name":"session_id","required":false,"type":"string"},{"id":12,"name":"user_id","required":false,"type":"string"},{"id":13,"name":"ua","required":false,"type":"string"},{"id":14,"name":"visits_id","required":false,"type":"string"},{"id":15,"name":"first_visit_time","required":false,"type":"string"},{"id":16,"name":"first_visit_load_id","required":false,"type":"string"},{"id":17,"name":"first_visit_ref_domain","required":false,"type":"string"},{"id":18,"name":"first_visit_subdomain_time","required":false,"type":"string"},{"id":19,"name":"first_visit_subdomain_load_id","required":false,"type":"string"},{"id":20,"name":"first_visit_subdomain_refer","required":false,"type":"string"},{"id":21,"name":"visits_ref","required":false,"type":"string"},{"id":22,"name":"visits_keyword","required":false,"type":"string"},{"id":23,"name":"visits_area_id","required":false,"type":"string"},{"id":24,"name":"bt_visits_ref","required":false,"type":"string"},{"id":25,"name":"last_visit_subdomain_refer","required":false,"type":"string"},{"id":26,"name":"site_ref","required":false,"type":"string"},{"id":27,"name":"scene_type","required":false,"type":"string"},{"id":28,"name":"show_id","required":false,"type":"string"},{"id":29,"name":"ref","required":false,"type":"string"},{"id":30,"name":"refer_domain","required":false,"type":"string"},{"id":31,"name":"key_word","required":false,"type":"string"},{"id":32,"name":"area_id","required":false,"type":"string"},{"id":33,"name":"current_url","required":false,"type":"string"},{"id":34,"name":"current_domain","required":false,"type":"string"},{"id":35,"name":"site_id","required":false,"type":"string"},{"id":36,"name":"category_id","required":false,"type":"string"},{"id":37,"name":"sub_category_id","required":false,"type":"string"},{"id":38,"name":"object_id","required":false,"type":"string"},{"id":39,"name":"series_id","required":false,"type":"string"},{"id":40,"name":"spec_id","required":false,"type":"string"},{"id":41,"name":"dealer_id","required":false,"type":"string"},{"id":42,"name":"level_id","required":false,"type":"string"},{"id":43,"name":"doc_title","required":false,"type":"string"},{"id":44,"name":"page_custom_vars","required":false,"type":"string"},{"id":45,"name":"platform","required":false,"type":"string"},{"id":46,"name":"screen","required":false,"type":"string"},{"id":47,"name":"color_depth","required":false,"type":"string"},{"id":48,"name":"browser_charset","required":false,"type":"string"},{"id":49,"name":"browser_language","required":false,"type":"string"},{"id":50,"name":"browser_cookie_enable","required":false,"type":"string"},{"id":51,"name":"extend","required":false,"type":"string"},{"id":52,"name":"create_time","required":false,"type":"string"},{"id":53,"name":"id","required":false,"type":"string"},{"id":54,"name":"custom1","required":false,"type":"string"},{"id":55,"name":"custom2","required":false,"type":"string"},{"id":56,"name":"customer3","required":false,"type":"string"},{"id":57,"name":"customer4","required":false,"type":"string"},{"id":58,"name":"longitude","required":false,"type":"string"},{"id":59,"name":"latitude","required":false,"type":"string"},{"id":60,"name":"browse_type","required":false,"type":"string"},{"id":61,"name":"ac_userid","required":false,"type":"string"},{"id":62,"name":"ac_app_key","required":false,"type":"string"},{"id":63,"name":"ac_app_platform","required":false,"type":"string"},{"id":64,"name":"ac_app_ver","required":false,"type":"string"},{"id":65,"name":"ac_app_deviceid","required":false,"type":"string"},{"id":66,"name":"cur_visit_ref_domain","required":false,"type":"string"},{"id":67,"name":"cookie_content","required":false,"type":"string"},{"id":68,"name":"pre_visits_ref","required":false,"type":"string"},{"id":69,"name":"lt_visits_ref","required":false,"type":"string"},{"id":70,"name":"abtest","required":false,"type":"string"},{"id":71,"name":"wx_openid","required":false,"type":"string"},{"id":72,"name":"wx_source","required":false,"type":"string"},{"id":73,"name":"log_src","required":false,"type":"string"},{"id":74,"name":"dt","required":false,"type":"string"},{"id":75,"name":"hour","required":false,"type":"string"}]}2021-02-02 15:57:37,974 INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe: selectedColumns 1, [wx_source]2021-02-02 15:57:37,974 INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe: Using projectedSchema {"type":"struct","fields":[{"id":72,"name":"wx_source","required":false,"type":"string"}]}2021-02-02 15:57:37,998 INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe: serDeProperties : {columns.types=string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string, location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1, columns=version,load_id,pre_load_id,sign,visit_time,ip,province_id,city_id,county_id,appkey,session_id,user_id,ua,visits_id,first_visit_time,first_visit_load_id,first_visit_ref_domain,first_visit_subdomain_time,first_visit_subdomain_load_id,first_visit_subdomain_refer,visits_ref,visits_keyword,visits_area_id,bt_visits_ref,last_visit_subdomain_refer,site_ref,scene_type,show_id,ref,refer_domain,key_word,area_id,current_url,current_domain,site_id,category_id,sub_category_id,object_id,series_id,spec_id,dealer_id,level_id,doc_title,page_custom_vars,platform,screen,color_depth,browser_charset,browser_language,browser_cookie_enable,extend,create_time,id,custom1,custom2,customer3,customer4,longitude,latitude,browse_type,ac_userid,ac_app_key,ac_app_platform,ac_app_ver,ac_app_deviceid,cur_visit_ref_domain,cookie_content,pre_visits_ref,lt_visits_ref,abtest,wx_openid,wx_source,log_src,dt,hour, metadata_location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1/metadata/00753-b2322cf0-ac32-4eb0-957e-95d2189e0272.metadata.json, previous_metadata_location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1/metadata/00752-6949a171-403e-4853-93a3-666df4305603.metadata.json, numFiles=59177, serialization.ddl=struct icebert_table1 { string version, string load_id, string pre_load_id, string sign, string visit_time, string ip, string province_id, string city_id, string county_id, string appkey, string session_id, string user_id, string ua, string visits_id, string first_visit_time, string first_visit_load_id, string first_visit_ref_domain, string first_visit_subdomain_time, string first_visit_subdomain_load_id, string first_visit_subdomain_refer, string visits_ref, string visits_keyword, string visits_area_id, string bt_visits_ref, string last_visit_subdomain_refer, string site_ref, string scene_type, string show_id, string ref, string refer_domain, string key_word, string area_id, string current_url, string current_domain, string site_id, string category_id, string sub_category_id, string object_id, string series_id, string spec_id, string dealer_id, string level_id, string doc_title, string page_custom_vars, string platform, string screen, string color_depth, string browser_charset, string browser_language, string browser_cookie_enable, string extend, string create_time, string id, string custom1, string custom2, string customer3, string customer4, string longitude, string latitude, string browse_type, string ac_userid, string ac_app_key, string ac_app_platform, string ac_app_ver, string ac_app_deviceid, string cur_visit_ref_domain, string cookie_content, string pre_visits_ref, string lt_visits_ref, string abtest, string wx_openid, string wx_source, string log_src, string dt, string hour}, transient_lastDdlTime=1611904166, iceberg.mr.table.schema={"type":"struct","fields":[{"id":1,"name":"version","required":false,"type":"string"},{"id":2,"name":"load_id","required":false,"type":"string"},{"id":3,"name":"pre_load_id","required":false,"type":"string"},{"id":4,"name":"sign","required":false,"type":"string"},{"id":5,"name":"visit_time","required":false,"type":"string"},{"id":6,"name":"ip","required":false,"type":"string"},{"id":7,"name":"province_id","required":false,"type":"string"},{"id":8,"name":"city_id","required":false,"type":"string"},{"id":9,"name":"county_id","required":false,"type":"string"},{"id":10,"name":"appkey","required":false,"type":"string"},{"id":11,"name":"session_id","required":false,"type":"string"},{"id":12,"name":"user_id","required":false,"type":"string"},{"id":13,"name":"ua","required":false,"type":"string"},{"id":14,"name":"visits_id","required":false,"type":"string"},{"id":15,"name":"first_visit_time","required":false,"type":"string"},{"id":16,"name":"first_visit_load_id","required":false,"type":"string"},{"id":17,"name":"first_visit_ref_domain","required":false,"type":"string"},{"id":18,"name":"first_visit_subdomain_time","required":false,"type":"string"},{"id":19,"name":"first_visit_subdomain_load_id","required":false,"type":"string"},{"id":20,"name":"first_visit_subdomain_refer","required":false,"type":"string"},{"id":21,"name":"visits_ref","required":false,"type":"string"},{"id":22,"name":"visits_keyword","required":false,"type":"string"},{"id":23,"name":"visits_area_id","required":false,"type":"string"},{"id":24,"name":"bt_visits_ref","required":false,"type":"string"},{"id":25,"name":"last_visit_subdomain_refer","required":false,"type":"string"},{"id":26,"name":"site_ref","required":false,"type":"string"},{"id":27,"name":"scene_type","required":false,"type":"string"},{"id":28,"name":"show_id","required":false,"type":"string"},{"id":29,"name":"ref","required":false,"type":"string"},{"id":30,"name":"refer_domain","required":false,"type":"string"},{"id":31,"name":"key_word","required":false,"type":"string"},{"id":32,"name":"area_id","required":false,"type":"string"},{"id":33,"name":"current_url","required":false,"type":"string"},{"id":34,"name":"current_domain","required":false,"type":"string"},{"id":35,"name":"site_id","required":false,"type":"string"},{"id":36,"name":"category_id","required":false,"type":"string"},{"id":37,"name":"sub_category_id","required":false,"type":"string"},{"id":38,"name":"object_id","required":false,"type":"string"},{"id":39,"name":"series_id","required":false,"type":"string"},{"id":40,"name":"spec_id","required":false,"type":"string"},{"id":41,"name":"dealer_id","required":false,"type":"string"},{"id":42,"name":"level_id","required":false,"type":"string"},{"id":43,"name":"doc_title","required":false,"type":"string"},{"id":44,"name":"page_custom_vars","required":false,"type":"string"},{"id":45,"name":"platform","required":false,"type":"string"},{"id":46,"name":"screen","required":false,"type":"string"},{"id":47,"name":"color_depth","required":false,"type":"string"},{"id":48,"name":"browser_charset","required":false,"type":"string"},{"id":49,"name":"browser_language","required":false,"type":"string"},{"id":50,"name":"browser_cookie_enable","required":false,"type":"string"},{"id":51,"name":"extend","required":false,"type":"string"},{"id":52,"name":"create_time","required":false,"type":"string"},{"id":53,"name":"id","required":false,"type":"string"},{"id":54,"name":"custom1","required":false,"type":"string"},{"id":55,"name":"custom2","required":false,"type":"string"},{"id":56,"name":"customer3","required":false,"type":"string"},{"id":57,"name":"customer4","required":false,"type":"string"},{"id":58,"name":"longitude","required":false,"type":"string"},{"id":59,"name":"latitude","required":false,"type":"string"},{"id":60,"name":"browse_type","required":false,"type":"string"},{"id":61,"name":"ac_userid","required":false,"type":"string"},{"id":62,"name":"ac_app_key","required":false,"type":"string"},{"id":63,"name":"ac_app_platform","required":false,"type":"string"},{"id":64,"name":"ac_app_ver","required":false,"type":"string"},{"id":65,"name":"ac_app_deviceid","required":false,"type":"string"},{"id":66,"name":"cur_visit_ref_domain","required":false,"type":"string"},{"id":67,"name":"cookie_content","required":false,"type":"string"},{"id":68,"name":"pre_visits_ref","required":false,"type":"string"},{"id":69,"name":"lt_visits_ref","required":false,"type":"string"},{"id":70,"name":"abtest","required":false,"type":"string"},{"id":71,"name":"wx_openid","required":false,"type":"string"},{"id":72,"name":"wx_source","required":false,"type":"string"},{"id":73,"name":"log_src","required":false,"type":"string"},{"id":74,"name":"dt","required":false,"type":"string"},{"id":75,"name":"hour","required":false,"type":"string"}]}, columns.comments=, totalSize=302458258710, bucket_count=0, file.outputformat=org.apache.iceberg.mr.hive.HiveIcebergOutputFormat, serialization.lib=org.apache.iceberg.mr.hive.HiveIcebergSerDe, storage_handler=org.apache.iceberg.mr.hive.HiveIcebergStorageHandler, table_type=ICEBERG, file.inputformat=org.apache.iceberg.mr.hive.HiveIcebergInputFormat, EXTERNAL=TRUE, name=flink_fdm_iceberg.icebert_table1}2021-02-02 15:57:37,999 INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe: Using schema from serDeProperties {"type":"struct","fields":[{"id":1,"name":"version","required":false,"type":"string"},{"id":2,"name":"load_id","required":false,"type":"string"},{"id":3,"name":"pre_load_id","required":false,"type":"string"},{"id":4,"name":"sign","required":false,"type":"string"},{"id":5,"name":"visit_time","required":false,"type":"string"},{"id":6,"name":"ip","required":false,"type":"string"},{"id":7,"name":"province_id","required":false,"type":"string"},{"id":8,"name":"city_id","required":false,"type":"string"},{"id":9,"name":"county_id","required":false,"type":"string"},{"id":10,"name":"appkey","required":false,"type":"string"},{"id":11,"name":"session_id","required":false,"type":"string"},{"id":12,"name":"user_id","required":false,"type":"string"},{"id":13,"name":"ua","required":false,"type":"string"},{"id":14,"name":"visits_id","required":false,"type":"string"},{"id":15,"name":"first_visit_time","required":false,"type":"string"},{"id":16,"name":"first_visit_load_id","required":false,"type":"string"},{"id":17,"name":"first_visit_ref_domain","required":false,"type":"string"},{"id":18,"name":"first_visit_subdomain_time","required":false,"type":"string"},{"id":19,"name":"first_visit_subdomain_load_id","required":false,"type":"string"},{"id":20,"name":"first_visit_subdomain_refer","required":false,"type":"string"},{"id":21,"name":"visits_ref","required":false,"type":"string"},{"id":22,"name":"visits_keyword","required":false,"type":"string"},{"id":23,"name":"visits_area_id","required":false,"type":"string"},{"id":24,"name":"bt_visits_ref","required":false,"type":"string"},{"id":25,"name":"last_visit_subdomain_refer","required":false,"type":"string"},{"id":26,"name":"site_ref","required":false,"type":"string"},{"id":27,"name":"scene_type","required":false,"type":"string"},{"id":28,"name":"show_id","required":false,"type":"string"},{"id":29,"name":"ref","required":false,"type":"string"},{"id":30,"name":"refer_domain","required":false,"type":"string"},{"id":31,"name":"key_word","required":false,"type":"string"},{"id":32,"name":"area_id","required":false,"type":"string"},{"id":33,"name":"current_url","required":false,"type":"string"},{"id":34,"name":"current_domain","required":false,"type":"string"},{"id":35,"name":"site_id","required":false,"type":"string"},{"id":36,"name":"category_id","required":false,"type":"string"},{"id":37,"name":"sub_category_id","required":false,"type":"string"},{"id":38,"name":"object_id","required":false,"type":"string"},{"id":39,"name":"series_id","required":false,"type":"string"},{"id":40,"name":"spec_id","required":false,"type":"string"},{"id":41,"name":"dealer_id","required":false,"type":"string"},{"id":42,"name":"level_id","required":false,"type":"string"},{"id":43,"name":"doc_title","required":false,"type":"string"},{"id":44,"name":"page_custom_vars","required":false,"type":"string"},{"id":45,"name":"platform","required":false,"type":"string"},{"id":46,"name":"screen","required":false,"type":"string"},{"id":47,"name":"color_depth","required":false,"type":"string"},{"id":48,"name":"browser_charset","required":false,"type":"string"},{"id":49,"name":"browser_language","required":false,"type":"string"},{"id":50,"name":"browser_cookie_enable","required":false,"type":"string"},{"id":51,"name":"extend","required":false,"type":"string"},{"id":52,"name":"create_time","required":false,"type":"string"},{"id":53,"name":"id","required":false,"type":"string"},{"id":54,"name":"custom1","required":false,"type":"string"},{"id":55,"name":"custom2","required":false,"type":"string"},{"id":56,"name":"customer3","required":false,"type":"string"},{"id":57,"name":"customer4","required":false,"type":"string"},{"id":58,"name":"longitude","required":false,"type":"string"},{"id":59,"name":"latitude","required":false,"type":"string"},{"id":60,"name":"browse_type","required":false,"type":"string"},{"id":61,"name":"ac_userid","required":false,"type":"string"},{"id":62,"name":"ac_app_key","required":false,"type":"string"},{"id":63,"name":"ac_app_platform","required":false,"type":"string"},{"id":64,"name":"ac_app_ver","required":false,"type":"string"},{"id":65,"name":"ac_app_deviceid","required":false,"type":"string"},{"id":66,"name":"cur_visit_ref_domain","required":false,"type":"string"},{"id":67,"name":"cookie_content","required":false,"type":"string"},{"id":68,"name":"pre_visits_ref","required":false,"type":"string"},{"id":69,"name":"lt_visits_ref","required":false,"type":"string"},{"id":70,"name":"abtest","required":false,"type":"string"},{"id":71,"name":"wx_openid","required":false,"type":"string"},{"id":72,"name":"wx_source","required":false,"type":"string"},{"id":73,"name":"log_src","required":false,"type":"string"},{"id":74,"name":"dt","required":false,"type":"string"},{"id":75,"name":"hour","required":false,"type":"string"}]}2021-02-02 15:57:37,999 INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe: selectedColumns 1, [wx_source]2021-02-02 15:57:38,000 INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe: Using projectedSchema {"type":"struct","fields":[{"id":72,"name":"wx_source","required":false,"type":"string"}]}2021-02-02 15:57:38,005 INFO [main] org.apache.hadoop.hive.ql.exec.mr.ExecMapper: <MAP>Id =0  <Children>null    <TS>Id =3      <Children>        <FIL>Id =18          <Children>            <SEL>Id =5              <Children>                <RS>Id =7                  <Children>                  <\Children>                  <Parent>Id = 5 null<\Parent>                <\RS>              <\Children>              <Parent>Id = 18 null<\Parent>            <\SEL>          <\Children>          <Parent>Id = 3 null<\Parent>        <\FIL>      <\Children>      <Parent>Id = 0 null<\Parent>    <\TS>  <\Children>  <Parent><\Parent><\MAP>2021-02-02 15:57:38,007 INFO [main] org.apache.hadoop.hive.ql.exec.TableScanOperator: Initializing operator TS[3]2021-02-02 15:57:38,008 INFO [main] org.apache.hadoop.hive.ql.exec.FilterOperator: Initializing operator FIL[18]2021-02-02 15:57:43,031 INFO [main] org.apache.hadoop.hive.ql.exec.SelectOperator: Initializing operator SEL[5]2021-02-02 15:57:43,031 INFO [main] org.apache.hadoop.hive.ql.exec.SelectOperator: SELECT struct<version:string,load_id:string,pre_load_id:string,sign:string,visit_time:string,ip:string,province_id:int,city_id:int,county_id:string,appkey:string,session_id:string,user_id:int,ua:string,visits_id:string,first_visit_time:string,first_visit_load_id:string,first_visit_ref_domain:string,first_visit_subdomain_time:string,first_visit_subdomain_load_id:string,first_visit_subdomain_refer:string,visits_ref:string,visits_keyword:string,visits_area_id:string,bt_visits_ref:string,last_visit_subdomain_refer:string,site_ref:string,scene_type:string,show_id:string,ref:string,refer_domain:string,key_word:string,area_id:string,current_url:string,current_domain:string,site_id:int,category_id:int,sub_category_id:int,object_id:int,series_id:int,spec_id:int,dealer_id:int,level_id:int,doc_title:string,page_custom_vars:string,platform:int,screen:string,color_depth:string,browser_charset:string,browser_language:string,browser_cookie_enable:string,extend:string,create_time:string,id:string,custom1:int,custom2:int,customer3:string,customer4:string,longitude:float,latitude:float,browse_type:string,ac_userid:string,ac_app_key:string,ac_app_platform:string,ac_app_ver:string,ac_app_deviceid:string,cur_visit_ref_domain:string,cookie_content:string,pre_visits_ref:string,lt_visits_ref:string,abtest:string,wx_openid:string,wx_source:string,log_src:string,dt:string,hour:string>2021-02-02 15:57:43,033 INFO [main] org.apache.hadoop.hive.ql.exec.ReduceSinkOperator: Initializing operator RS[7]2021-02-02 15:57:43,036 INFO [main] org.apache.hadoop.hive.ql.exec.ReduceSinkOperator: Using tag = 12021-02-02 15:57:43,050 INFO [main] org.apache.hadoop.hive.ql.exec.TableScanOperator: Initializing operator TS[0]2021-02-02 15:57:43,050 INFO [main] org.apache.hadoop.hive.ql.exec.FilterOperator: Initializing operator FIL[17]2021-02-02 15:57:43,051 INFO [main] org.apache.hadoop.hive.ql.exec.SelectOperator: Initializing operator SEL[2]2021-02-02 15:57:43,051 INFO [main] org.apache.hadoop.hive.ql.exec.SelectOperator: SELECT struct<wx_source:string>2021-02-02 15:57:43,052 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) ... 9 moreCaused by: java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38) ... 14 moreCaused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) ... 17 moreCaused by: java.lang.RuntimeException: Map operator initialization failed at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:136) ... 22 moreCaused by: java.lang.RuntimeException: cannot find field log_src from [org.apache.iceberg.mr.hive.serde.objectinspector.IcebergRecordObjectInspector$IcebergRecordStructField@8838736] at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:442) at org.apache.iceberg.mr.hive.serde.objectinspector.IcebergRecordObjectInspector.getStructFieldRef(IcebergRecordObjectInspector.java:78) at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:55) at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:139) at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:980) at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:1006) at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:77) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:355) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:504) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:457) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:365) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:504) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:457) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:365) at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:498) at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:115) ... 22 more
2021-02-02 15:57:43,053 INFO [main] org.apache.hadoop.mapred.Task: Runnning cleanup for the task

Attachments

Activity

People

Assignee:: Unassigned

Reporter:: Xingxing Di

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 02/Feb/21 11:35

Updated:: 11/May/21 03:30