[HIVE-27749] SchemaTool initSchema fails on Mariadb 10.2

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 4.0.0-alpha-2, 4.0.0-beta-1
• Fix Version/s: 4.0.0
• Component/s: Standalone Metastore
• Labels: pull-request-available
• Assignee: Sourabh Badhya
• Reporter: Stamatis Zampetakis
• Created: 27/Sep/23 19:50
• Updated: 29/Mar/24 17:49
• Resolved: 18/Jan/24 08:10

Description

Schema initialization for 4.0.0-beta-1 fails when run on Mariadb 10.2.

The problem is reproducible on current (e5a7ce2f091da1f8a324da6e489cda59b9e4bfc6) master by applying the mariadb-metastore-schema-tests.patch and then running:

mvn test -Dtest=TestMariadb#install -Dtest.groups=""

The error is shown below:

315/409      ALTER TABLE `NOTIFICATION_SEQUENCE` MODIFY COLUMN `NNI_ID` BIGINT(20) GENERATED ALWAYS AS (1) STORED NOT NULL;
Error: (conn=11) You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'NOT NULL' at line 1 (state=42000,code=1064)
Aborting command set because "force" is false and command failed: "ALTER TABLE `NOTIFICATION_SEQUENCE` MODIFY COLUMN `NNI_ID` BIGINT(20) GENERATED ALWAYS AS (1) STORED NOT NULL;"
[ERROR] 2023-09-27 21:36:30.317 [main] MetastoreSchemaTool - Schema initialization FAILED! Metastore state would be inconsistent!
Schema initialization FAILED! Metastore state would be inconsistent!
[ERROR] 2023-09-27 21:36:30.317 [main] MetastoreSchemaTool - Underlying cause: java.io.IOException : Schema script failed, errorcode OTHER
Underlying cause: java.io.IOException : Schema script failed, errorcode OTHER
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent!
        at org.apache.hadoop.hive.metastore.tools.schematool.SchemaToolTaskInit.execute(SchemaToolTaskInit.java:66)
        at org.apache.hadoop.hive.metastore.tools.schematool.MetastoreSchemaTool.run(MetastoreSchemaTool.java:480)
        at org.apache.hadoop.hive.metastore.tools.schematool.MetastoreSchemaTool.run(MetastoreSchemaTool.java:425)
        at org.apache.hadoop.hive.metastore.dbinstall.rules.DatabaseRule.installLatest(DatabaseRule.java:269)
        at org.apache.hadoop.hive.metastore.dbinstall.DbInstallBase.install(DbInstallBase.java:34)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
        at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
        at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
        at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
        at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
        at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377)
        at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138)
        at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451)
Caused by: java.io.IOException: Schema script failed, errorcode OTHER
        at org.apache.hadoop.hive.metastore.tools.schematool.MetastoreSchemaTool.execSql(MetastoreSchemaTool.java:332)
        at org.apache.hadoop.hive.metastore.tools.schematool.MetastoreSchemaTool.execSql(MetastoreSchemaTool.java:303)
        at org.apache.hadoop.hive.metastore.tools.schematool.SchemaToolTaskInit.execute(SchemaToolTaskInit.java:62)
        ... 33 more
[ERROR] 2023-09-27 21:36:30.317 [main] MetastoreSchemaTool - *** schemaTool failed ***
*** schemaTool failed ***

Issue Links

• links to: GitHub Pull Request #4949
• links to: GitHub Pull Request #5009