[ATLAS-4187] Jackson-databind version 2.10.0 doesn't match with Hive version 3.1.2 on EMR 6.X - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Not A Problem
Affects Version/s: 2.1.0
Fix Version/s: 2.1.0
Component/s: hive-integration
Labels:
None

Description

After create the link jackson-databind-2.10.0.jar to /usr/lib/hive/lib hiveserver2 stops to work and return this log:

2021-03-05T20:44:58,774 INFO [main([])]: server.HiveServer2 (HiveServer2.java:startHiveServer2(1020)) - Starting HiveServer2
2021-03-05T20:44:58,893 INFO [main([])]: SessionState ()) - Hive Session ID = 1c43f577-8747-4058-b96f-dbf15e388f87
2021-03-05T20:44:59,842 INFO [main([])]: session.SessionState ()) - Created HDFS directory: /tmp/hive/hive/1c43f577-8747-4058-b96f-dbf15e388f87
2021-03-05T20:44:59,868 INFO [main([])]: session.SessionState ()) - Created local directory: /mnt/tmp/hive/1c43f577-8747-4058-b96f-dbf15e388f87
2021-03-05T20:44:59,871 INFO [main([])]: session.SessionState ()) - Created HDFS directory: /tmp/hive/hive/1c43f577-8747-4058-b96f-dbf15e388f87/_tmp_space.db
2021-03-05T20:44:59,892 INFO [main([])]: sqlstd.SQLStdHiveAccessController ()) - Created SQLStdHiveAccessController for session context : HiveAuthzSessionContext [sessionString=1c43f577-8747-4058-b96f-dbf15e388f87, clientType=HIVESERVER2]
2021-03-05T20:44:59,896 WARN [main([])]: session.SessionState ()) - METASTORE_FILTER_HOOK will be ignored, since hive.security.authorization.manager is set to instance of HiveAuthorizerFactory.
2021-03-05T20:45:00,660 INFO [main([])]: server.HiveServer2 (HiveServer2.java:stop(900)) - Shutting down HiveServer2
2021-03-05T20:45:00,661 INFO [main([])]: server.HiveServer2 (HiveServer2.java:stopOrDisconnectTezSessions(877)) - Stopping/Disconnecting tez sessions.
2021-03-05T20:45:00,661 WARN [main([])]: server.HiveServer2 (HiveServer2.java:startHiveServer2(1064)) - Error starting HiveServer2 on attempt 1, will retry in 60000ms
java.lang.VerifyError: Stack map does not match the one at exception handler 77
Exception Details:
Location:
com/fasterxml/jackson/databind/deser/std/StdDeserializer._parseDate(Lcom/fasterxml/jackson/core/JsonParser;Lcom/fasterxml/jackson/databind/DeserializationContext;)Ljava/util/Date; @77: astore
Reason:
Type 'com/fasterxml/jackson/core/JsonParseException' (current frame, stack[0]) is not assignable to 'com/fasterxml/jackson/core/exc/StreamReadException' (stack map, stack[0])
Current Frame:
bci: @69
flags: { }
locals: { 'com/fasterxml/jackson/databind/deser/std/StdDeserializer', 'com/fasterxml/jackson/core/JsonParser', 'com/fasterxml/jackson/databind/DeserializationContext' }
stack: { 'com/fasterxml/jackson/core/JsonParseException' }
Stackmap Frame:
bci: @77
flags: { }
locals: { 'com/fasterxml/jackson/databind/deser/std/StdDeserializer', 'com/fasterxml/jackson/core/JsonParser', 'com/fasterxml/jackson/databind/DeserializationContext' }
stack: { 'com/fasterxml/jackson/core/exc/StreamReadException' }
Bytecode:
0x0000000: 2bb6 0035 aa00 0000 0000 0081 0000 0003
0x0000010: 0000 000b 0000 007a 0000 0081 0000 0081
0x0000020: 0000 0034 0000 0041 0000 0081 0000 0081
0x0000030: 0000 0081 0000 0071 2a2b b600 11b6 0012
0x0000040: 2cb6 006b b02b b600 4742 a700 223a 052c
0x0000050: 2ab4 0002 2bb6 006e 126f 03bd 0004 b600
0x0000060: 70c0 002d 3a06 1906 b600 4c42 bb00 7159
0x0000070: 21b7 0072 b02a 2cb6 0073 c000 71b0 2a2b
0x0000080: 2cb6 0074 b02c 2ab4 0002 2bb6 0025 c000
0x0000090: 71b0
Exception Handler Table:
bci [69, 74] => handler: 77
bci [69, 74] => handler: 77
Stackmap Table:
same_frame(@56)
same_frame(@69)
same_locals_1_stack_item_frame(@77,Object359)
append_frame(@108,Long)
chop_frame(@117,1)
same_frame(@126)
same_frame(@133)

at com.fasterxml.jackson.databind.deser.std.JdkDeserializers.<clinit>(JdkDeserializers.java:26) ~[jackson-databind-2.10.0.jar:2.10.0]
at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.findDefaultDeserializer(BasicDeserializerFactory.java:1852) ~[jackson-databind-2.10.0.jar:2.10.0]
at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.findStdDeserializer(BeanDeserializerFactory.java:167) ~[jackson-databind-2.10.0.jar:2.10.0]
at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:131) ~[jackson-databind-2.10.0.jar:2.10.0]
at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:411) ~[jackson-databind-2.10.0.jar:2.10.0]
at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:349) ~[jackson-databind-2.10.0.jar:2.10.0]
at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:264) ~[jackson-databind-2.10.0.jar:2.10.0]
at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244) ~[jackson-databind-2.10.0.jar:2.10.0]
at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142) ~[jackson-databind-2.10.0.jar:2.10.0]
at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:476) ~[jackson-databind-2.10.0.jar:2.10.0]
at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:4389) ~[jackson-databind-2.10.0.jar:2.10.0]
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4198) ~[jackson-databind-2.10.0.jar:2.10.0]
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3136) ~[jackson-databind-2.10.0.jar:2.10.0]
at com.amazonaws.internal.config.InternalConfig.loadfrom(InternalConfig.java:250) ~[aws-java-sdk-core-1.11.828.jar:?]
at com.amazonaws.internal.config.InternalConfig.load(InternalConfig.java:263) ~[aws-java-sdk-core-1.11.828.jar:?]
at com.amazonaws.internal.config.InternalConfig$Factory.<clinit>(InternalConfig.java:336) ~[aws-java-sdk-core-1.11.828.jar:?]
at com.amazonaws.util.VersionInfoUtils.userAgent(VersionInfoUtils.java:142) ~[aws-java-sdk-core-1.11.828.jar:?]
at com.amazonaws.util.VersionInfoUtils.initializeUserAgent(VersionInfoUtils.java:137) ~[aws-java-sdk-core-1.11.828.jar:?]
at com.amazonaws.util.VersionInfoUtils.getUserAgent(VersionInfoUtils.java:100) ~[aws-java-sdk-core-1.11.828.jar:?]
at com.amazonaws.internal.EC2ResourceFetcher.<clinit>(EC2ResourceFetcher.java:44) ~[aws-java-sdk-core-1.11.828.jar:?]
at com.amazonaws.auth.InstanceMetadataServiceCredentialsFetcher.<init>(InstanceMetadataServiceCredentialsFetcher.java:36) ~[aws-java-sdk-core-1.11.828.jar:?]
at com.amazonaws.auth.InstanceProfileCredentialsProvider.<init>(InstanceProfileCredentialsProvider.java:102) ~[aws-java-sdk-core-1.11.828.jar:?]
at com.amazonaws.auth.InstanceProfileCredentialsProvider.<init>(InstanceProfileCredentialsProvider.java:82) ~[aws-java-sdk-core-1.11.828.jar:?]
at com.amazonaws.auth.InstanceProfileCredentialsProvider.<init>(InstanceProfileCredentialsProvider.java:66) ~[aws-java-sdk-core-1.11.828.jar:?]
at com.amazonaws.auth.InstanceProfileCredentialsProvider.<clinit>(InstanceProfileCredentialsProvider.java:49) ~[aws-java-sdk-core-1.11.828.jar:?]
at com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper.initializeProvider(EC2ContainerCredentialsProviderWrapper.java:64) ~[aws-java-sdk-core-1.11.828.jar:?]
at com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper.<init>(EC2ContainerCredentialsProviderWrapper.java:53) ~[aws-java-sdk-core-1.11.828.jar:?]
at com.amazonaws.auth.DefaultAWSCredentialsProviderChain.<init>(DefaultAWSCredentialsProviderChain.java:49) ~[aws-java-sdk-core-1.11.828.jar:?]
at com.amazonaws.auth.DefaultAWSCredentialsProviderChain.<clinit>(DefaultAWSCredentialsProviderChain.java:43) ~[aws-java-sdk-core-1.11.828.jar:?]
at com.amazonaws.glue.catalog.metastore.DefaultAWSCredentialsProviderFactory.buildAWSCredentialsProvider(DefaultAWSCredentialsProviderFactory.java:13) ~[aws-glue-datacatalog-hive3-client-3.0.0.jar:?]
at com.amazonaws.glue.catalog.metastore.AWSGlueClientFactory.getAWSCredentialsProvider(AWSGlueClientFactory.java:101) ~[aws-glue-datacatalog-hive3-client-3.0.0.jar:?]
at com.amazonaws.glue.catalog.metastore.AWSGlueClientFactory.newClient(AWSGlueClientFactory.java:49) ~[aws-glue-datacatalog-hive3-client-3.0.0.jar:?]
at com.amazonaws.glue.catalog.metastore.AWSCatalogMetastoreClient.<init>(AWSCatalogMetastoreClient.java:184) ~[aws-glue-datacatalog-hive3-client-3.0.0.jar:?]
at com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory.createMetaStoreClient(AWSGlueDataCatalogHiveClientFactory.java:19) ~[aws-glue-datacatalog-hive3-client-3.0.0.jar:?]
at org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:481) ~[hive-exec-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4371) ~[hive-exec-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4351) ~[hive-exec-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:4607) ~[hive-exec-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:287) ~[hive-exec-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:270) ~[hive-exec-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:443) ~[hive-exec-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:371) ~[hive-exec-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:351) ~[hive-exec-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:327) ~[hive-exec-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:960) ~[hive-exec-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:924) ~[hive-exec-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hadoop.hive.ql.session.SessionState.applyAuthorizationPolicy(SessionState.java:1893) ~[hive-exec-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:131) ~[hive-service-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hive.service.cli.CLIService.init(CLIService.java:115) ~[hive-service-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hive.service.CompositeService.init(CompositeService.java:59) ~[hive-service-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:230) ~[hive-service-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1036) [hive-service-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hive.service.server.HiveServer2.access$1600(HiveServer2.java:140) [hive-service-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1305) [hive-service-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1149) [hive-service-3.1.2-amzn-2.jar:3.1.2-amzn-2]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_282]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_282]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_282]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_282]
at org.apache.hadoop.util.RunJar.run(RunJar.java:323) [hadoop-common-3.2.1-amzn-1.jar:?]
at org.apache.hadoop.util.RunJar.main(RunJar.java:236) [hadoop-common-3.2.1-amzn-1.jar:?]
2021-03-05T20:46:00,666 INFO [main([])]: server.HiveServer2 (HiveServer2.java:startHiveServer2(1020)) - Starting HiveServer2

Attachments

Activity

People

Assignee:: Unassigned

Reporter:: Ricardo

Votes:: 0 Vote for this issue

Watchers:: 1 Start watching this issue

Dates

Created:: 05/Mar/21 20:51

Updated:: 05/Apr/21 18:20

Resolved:: 05/Apr/21 18:20